-- Logs begin at Sat 2022-05-28 02:37:38 UTC, end at Sat 2022-05-28 03:54:34 UTC. -- May 28 03:15:36.623214 dv-cinder-843361-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 28 03:15:36.631698 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat May 28 03:15:36 2022] *** May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: nodename: dv-cinder-843361-3-iscsi May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: machine: x86_64 May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: clock source: unix May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: pcre jit disabled May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: detected number of CPU cores: 4 May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: current working directory: / May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: detected binary path: /usr/bin/uwsgi-core May 28 03:15:36.633714 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: your processes number limit is 47680 May 28 03:15:36.634317 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: your memory page size is 4096 bytes May 28 03:15:36.634317 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: detected max file descriptor number: 2048 May 28 03:15:36.634317 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: lock engine: pthread robust mutexes May 28 03:15:36.634317 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: thunder lock: enabled May 28 03:15:36.634317 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 28 03:15:36.634317 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 28 03:15:36.678459 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: Python main interpreter initialized at 0x564af78499c0 May 28 03:15:36.678459 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: python threads support enabled May 28 03:15:36.678459 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: your server socket listen backlog is limited to 100 connections May 28 03:15:36.678459 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: your mercy for graceful operations on workers is 90 seconds May 28 03:15:36.678803 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: mapped 403077 bytes (393 KB) for 2 cores May 28 03:15:36.678877 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: *** Operational MODE: preforking *** May 28 03:15:36.679122 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: *** uWSGI is running in multiple interpreter mode *** May 28 03:15:36.679122 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: spawned uWSGI master process (pid: 80341) May 28 03:15:36.679317 dv-cinder-843361-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 28 03:15:36.680977 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: spawned uWSGI worker 1 (pid: 80354, cores: 1) May 28 03:15:36.680977 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: spawned uWSGI worker 2 (pid: 80355, cores: 1) May 28 03:15:36.680977 dv-cinder-843361-3-iscsi devstack@keystone.service[80341]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 28 03:15:42.326260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:15:42.326260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: from cryptography.utils import int_from_bytes May 28 03:15:42.327018 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:15:42.327018 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: from cryptography.utils import int_from_bytes May 28 03:15:42.327125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:15:42.327125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: from cryptography.utils import int_from_bytes May 28 03:15:42.327125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 03:15:42.327125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: from cryptography.utils import int_from_bytes May 28 03:15:42.376220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 03:15:42.376402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 03:15:42.376490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 03:15:42.376581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 03:15:42.376658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] command line args: [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 03:15:42.376733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] command line args: [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 03:15:42.376833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 03:15:42.376979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 03:15:42.377057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ================================================================================ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 03:15:42.377162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ================================================================================ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 03:15:42.377275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] admin_token = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.377430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] admin_token = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.377507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_dir = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.377675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] config_dir = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.377757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.377862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_source = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.377944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] config_source = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] debug = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] debug = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.378976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_config_append = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379753 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_config_append = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.379943 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_dir = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_dir = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_file = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.380990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.381893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.382947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.383032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.383189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.383272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.383456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.383536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.383938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] publish_errors = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] publish_errors = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.384986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.385077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.385242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.385317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.385466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] standard_threads = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.385539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] standard_threads = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.385690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.385771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.385929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_journal = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_journal = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_json = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_json = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_stderr = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_stderr = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] use_syslog = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_syslog = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.386988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.387063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 03:15:42.387238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.387320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.387474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.387548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.387726 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.387799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.387960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.388041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.388203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.388278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.388438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.388514 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.388698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.388826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.388900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.389017 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.389246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.389336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.389499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.389577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.389750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.389825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.389980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.390055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.390222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.390300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.390480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.390562 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.390722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.390882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.390956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.391025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.391185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.391261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.391433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.391517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.391680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.391757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.391921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392731 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392902 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.392987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.393169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.393248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.393398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.393473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.393627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.393715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.393850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.393931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.394083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.394157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.394326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.394411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.394572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.394648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.394797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.394872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395590 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.395986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.396994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.397119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.397200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.397353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.397430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.397581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.397652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.397813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.397891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.398056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.398132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.398288 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.398420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.398521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.398609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.398737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.398881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.399018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.399145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.399277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.399392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.399522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.399645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.399765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.399881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.400012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.400130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.400262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.400416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.400539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.400657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.400778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.400918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.401040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.401192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.401315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.401446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.401574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.401707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.401832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.401952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.402076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.402213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.402338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.402476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.402606 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.402806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.402938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.403069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.403195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.403324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.403412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.403580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.403705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.403829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.403960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.404086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.404218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.404343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.404466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.404591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.404714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.404843 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.404974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.405869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.406461 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.406542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.406613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.406681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.406749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.406816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.406883 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.406958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.407961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.408991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.409169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.409250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.409404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.409480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.409638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.409716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.409870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.409952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.410092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.410168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.410324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.410400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.410554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.410629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.410786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.410860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411018 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.411972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.external = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.412047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.412199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.412275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.external = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.412427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.412509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.412704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.412792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.412930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.password = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.password = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] auth.token = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.token = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.413957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.414105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.414188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.414346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.414422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.414591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.414667 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.414818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.414904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.415997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.416149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.416224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.416293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.416454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.416531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.416686 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.416770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.416949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.417025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.417208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.417300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.417447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.417532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.417712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.417790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.417949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.418024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.418205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.418289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.418442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.418519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.418692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.418769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.418922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.419947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.420100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.420175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.420335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.420418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.420569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.420645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.420803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.420879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.421046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.421148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.421315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.421392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.422136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.422233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.422362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.422447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.422606 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.422682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.422845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.422921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.423079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.423166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.423313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.423390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.423547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.423622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.423771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.423854 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.424016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.424092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.424256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.424333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.424505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.424716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.424795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.424981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.425091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.425229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.425417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.425507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.425649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.425726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.425892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.425969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.426135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.426221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.426379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.426456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.426627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.426703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.426867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.426945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.427173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.427262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.427361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.427438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.427591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.427670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.427816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.427893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.428973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.429052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.429242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.429321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.429473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.429553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.429715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.429794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.429955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.430970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.431121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.431199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.431344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.431420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.431574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.431651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.431805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.431885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.432970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.433046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.433222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.433302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.433450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.433528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.433691 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.433769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.433925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.434925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.435083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.435166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.435336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.435414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.435560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.435637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.435799 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.435876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436723 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.436952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.437030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.437253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.437333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.437489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.437568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.437714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.437791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.437956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.438988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.439133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.439210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.439374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.439523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.439598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.439668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.439824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.439902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.440991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.441069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.441252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.441330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.441476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.441564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.441715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.441793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.441948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.442026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.442179 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.442269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.442414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.442493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.442655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.442797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.442875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.443043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.443122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.443270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.443353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.443501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.443579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.443738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.443817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.444026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.444103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.444274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.444357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.444524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.444604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.444763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.444841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445510 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.445981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] role.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] role.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.446999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.447164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.447241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.447405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.447483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.447648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.447746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.447891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.447997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.448134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.448245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.448374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.448479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.448615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.448726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.448843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.448976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.449083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.449250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.449350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.449499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.449580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.449821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.449897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.450118 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.450194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.450369 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.450448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.450620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.450698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.450868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.450946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.451119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.451198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.451355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.451434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.451601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.451677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.451849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.451927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.452088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.452166 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.452321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.452397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.452550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.452630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.452804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.452887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.453044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.453139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.453321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.453402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.453567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.453644 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.453808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.453887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.454966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455282 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.455993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.456145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.456222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.456382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.456460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.456620 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.456698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.456854 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.456932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.457083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.457216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.457342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.457420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.457573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.457653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.457814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.457892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.458040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.458124 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.458267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.458344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.458503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.458581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.458791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.458877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.459980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.460981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.461155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.461235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.461391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.461470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.461624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.461701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.461852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.461929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.462092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.462170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.462321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.462397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.462551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.462628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.462774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.462852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.463010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.463088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.463178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 03:15:42.463255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 03:15:42.463397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80355) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 03:15:42.494677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.494907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.495125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.495208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.495352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.495571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.503816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.504017 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.532346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.532572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.532785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.533012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.584791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.585198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.636116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.636349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.636591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.636819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.701533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.701661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.701757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.701884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.701967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.702105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.702187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.702318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.702402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.702526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.702608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.702733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.726219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.726429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.726515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.726655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.726736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.726870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.726960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.727097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.727234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.727322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.727399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.727488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.727567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.727687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.739340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.739615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.751766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.752946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.753074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.753177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.753311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80355) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 03:15:42.771955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.772163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 03:15:42.772407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.772491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.772614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 03:15:42.772696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 03:15:42.772928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.773056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.773212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.773297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 03:15:42.773466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.773547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.773720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.773798 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.773967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.774215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.785197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.785472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.785668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.785762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.785936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.786017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.786197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.786469 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.794073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.794742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.797933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.798211 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.798490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.798571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.798769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.798850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.799045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.799125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.799299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.799389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.799600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.799680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.799853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.799932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.800184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.800264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.800455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.800534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.800723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.800856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.800989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.801255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.826928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.827249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.829216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.829492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.829630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.829755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.834759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.835024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.835166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.835292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.835415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.835537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.842629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.842763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.842899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.843039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.848203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.848338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.848594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.848733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.848859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.848983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.849149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.849277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.849446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.849573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.849707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.849832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.849956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.850087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.850211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.850335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.868591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.868844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.868973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.869134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.873536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.873789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.877182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.877471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.877736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.877872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.882655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.882785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.886237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.886378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.889976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.890143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.890790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.890967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.891105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.891230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.891355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.891517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.891643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.891802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.891940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.892074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.911409 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.911566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.911704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.911846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.911980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.912105 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.912237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.912362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.920122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.920305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.921859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.922054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.922182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.922325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.925160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.925389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.925472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.925668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.925751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.925948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.926029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.926215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.926303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.926487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.939688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.939925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.940009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.940202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.940285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.940476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.940557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.940747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.940837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.941019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.941117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.941312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.941392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.941592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.959780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.960083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.963040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.963340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.966335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.966663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.967919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.968261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.969636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.969913 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.970005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.970190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.970271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.970506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.978463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.980070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.980164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.983086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.983752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.984596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.984947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.985263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.988651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.988942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.989265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.989538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.994893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.995177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.995560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.998782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:42.999062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:42.999436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.004084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:43.004360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.004633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.004904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.005206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.005478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.005745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.006028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.006295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.006567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.006835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.007093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.007354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.007630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.007828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80355) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 03:15:43.008097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.008367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.008639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.008901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.009190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.009482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.009743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.010003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.010270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.010534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.010793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.011055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.011312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80355) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 03:15:43.051119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x564af78499c0 pid: 80355 (default app) May 28 03:15:43.052191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x564af78499c0 pid: 80354 (default app) May 28 03:15:43.054344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe6073-92bb-4205-bf88-01383f33f682 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:43.054635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe6073-92bb-4205-bf88-01383f33f682 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:43.054909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe6073-92bb-4205-bf88-01383f33f682 None None] PATH_INFO: `/v3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:43.059150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1/1] 10.222.0.41 () {50 vars in 791 bytes} [Sat May 28 03:15:43 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:15:51.938990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b0d5-fd83-4309-8851-115993212726 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:51.939463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b0d5-fd83-4309-8851-115993212726 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:51.939553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b0d5-fd83-4309-8851-115993212726 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:51.943916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1/2] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:15:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:15:51.949002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:51.949462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:51.949598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:52.017274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: INFO dbcounter [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] Registered counter for database keystone May 28 03:15:52.038575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG oslo_db.sqlalchemy.engines [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80355) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:15:52.044841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writer thread running {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:15:52.222253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG passlib.handlers.bcrypt [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80355) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 28 03:15:52.222596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG passlib.handlers.bcrypt [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80355) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 28 03:15:52.277184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:15:52.278793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:15:52.401742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 2/3] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:15:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 28 03:15:52.575633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b0d5-fd83-4309-8851-115993212726 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:52.644734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: INFO dbcounter [None req-4a47b0d5-fd83-4309-8851-115993212726 None None] Registered counter for database keystone May 28 03:15:52.664137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG oslo_db.sqlalchemy.engines [None req-4a47b0d5-fd83-4309-8851-115993212726 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80354) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 03:15:52.670320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writer thread running {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 03:15:52.789092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b0d5-fd83-4309-8851-115993212726 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:52.793391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed85fc5-267a-4d19-b57d-fbe22c27b4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:52.794045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed85fc5-267a-4d19-b57d-fbe22c27b4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:52.794331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed85fc5-267a-4d19-b57d-fbe22c27b4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:52.794611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed85fc5-267a-4d19-b57d-fbe22c27b4d8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:52.802705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed85fc5-267a-4d19-b57d-fbe22c27b4d8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:52.854342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bed85fc5-267a-4d19-b57d-fbe22c27b4d8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 28 03:15:52.854628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed85fc5-267a-4d19-b57d-fbe22c27b4d8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:52.864044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 2/4] 10.222.0.41 () {58 vars in 1324 bytes} [Sat May 28 03:15:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:15:52.870097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:52.884245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1105db9-ea57-4825-8e21-158add3e2649 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:52.888645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c072481-8623-4d73-a777-06619b8ee0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:52.889791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c072481-8623-4d73-a777-06619b8ee0ee admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:52.890585 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c072481-8623-4d73-a777-06619b8ee0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:52.890858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c072481-8623-4d73-a777-06619b8ee0ee admin admin] PATH_INFO: `/v3/projects/cc9d398b29a149a4ab99273f43b67fb2` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:52.892356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c072481-8623-4d73-a777-06619b8ee0ee admin admin] RBAC: Authorizing `identity:get_project(project_id=cc9d398b29a149a4ab99273f43b67fb2)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:53.050235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1c072481-8623-4d73-a777-06619b8ee0ee admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 28 03:15:53.050899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c072481-8623-4d73-a777-06619b8ee0ee admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:53.055228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 3/5] 10.222.0.41 () {56 vars in 1207 bytes} [Sat May 28 03:15:52 2022] GET /identity/v3/projects/cc9d398b29a149a4ab99273f43b67fb2 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:15:55.203586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73c855-664a-4201-82ee-f44d8a5580f1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:55.204085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73c855-664a-4201-82ee-f44d8a5580f1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:55.204085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73c855-664a-4201-82ee-f44d8a5580f1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:55.208193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 3/6] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:15:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:15:55.213530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a6a18-a0b2-486e-a436-e2c4fd6a4fdd None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:55.213905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a6a18-a0b2-486e-a436-e2c4fd6a4fdd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:55.214073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a6a18-a0b2-486e-a436-e2c4fd6a4fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:55.233279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4d8a6a18-a0b2-486e-a436-e2c4fd6a4fdd None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:15:55.235403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4d8a6a18-a0b2-486e-a436-e2c4fd6a4fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:15:55.243163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 4/7] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:15:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 28 03:15:55.264656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73c855-664a-4201-82ee-f44d8a5580f1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:55.276560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73c855-664a-4201-82ee-f44d8a5580f1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:55.281348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005eea8-509f-4bf4-9dcc-59711efbb109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:55.281971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005eea8-509f-4bf4-9dcc-59711efbb109 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:55.282257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005eea8-509f-4bf4-9dcc-59711efbb109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:55.282537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005eea8-509f-4bf4-9dcc-59711efbb109 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:55.290262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005eea8-509f-4bf4-9dcc-59711efbb109 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:55.292039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005eea8-509f-4bf4-9dcc-59711efbb109 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:55.301357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 4/8] 10.222.0.41 () {58 vars in 1324 bytes} [Sat May 28 03:15:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:15:55.306343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a6a18-a0b2-486e-a436-e2c4fd6a4fdd None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:55.315022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a6a18-a0b2-486e-a436-e2c4fd6a4fdd None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:55.319406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b8038-e7e6-40d9-a8c2-703d50b5fa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:55.320262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8038-e7e6-40d9-a8c2-703d50b5fa0b admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:55.320548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8038-e7e6-40d9-a8c2-703d50b5fa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:55.320827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b8038-e7e6-40d9-a8c2-703d50b5fa0b admin admin] PATH_INFO: `/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:55.322234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b8038-e7e6-40d9-a8c2-703d50b5fa0b admin admin] RBAC: Authorizing `identity:get_user(user_id=f454c88a6bba4ab9ba311bbbd1eb0425)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:55.324057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b8038-e7e6-40d9-a8c2-703d50b5fa0b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:55.325638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 5/9] 10.222.0.41 () {56 vars in 1198 bytes} [Sat May 28 03:15:55 2022] GET /identity/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:15:57.575022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ec0e2-6bdb-475e-8f45-8ba8d7f1e243 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:57.575629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ec0e2-6bdb-475e-8f45-8ba8d7f1e243 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:57.575629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ec0e2-6bdb-475e-8f45-8ba8d7f1e243 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:57.576934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 5/10] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:15:57.582736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b4f9f-043e-49f5-b404-2e7ca0307468 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:57.583157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b4f9f-043e-49f5-b404-2e7ca0307468 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:57.583417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b4f9f-043e-49f5-b404-2e7ca0307468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:57.599677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a16d93-77cc-4b49-9bc1-5558f361bc5c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:57.599969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a16d93-77cc-4b49-9bc1-5558f361bc5c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:57.600142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-644b4f9f-043e-49f5-b404-2e7ca0307468 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:15:57.600237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a16d93-77cc-4b49-9bc1-5558f361bc5c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:57.601346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-644b4f9f-043e-49f5-b404-2e7ca0307468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:15:57.601609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 6/11] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:15:57.606765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:57.607051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:57.607322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:57.646996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG passlib.handlers.bcrypt [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80354) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 28 03:15:57.647403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG passlib.handlers.bcrypt [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80354) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 28 03:15:57.680674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 6/12] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:15:57.693758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:15:57.695326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:15:57.704461 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b4f9f-043e-49f5-b404-2e7ca0307468 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:57.731003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:15:57.800112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 7/13] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:15:57.815175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b4f9f-043e-49f5-b404-2e7ca0307468 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:57.822274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:57.847694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b6aac-8e87-41e0-b261-bbf2950faf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:57.848419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6aac-8e87-41e0-b261-bbf2950faf0c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:57.848704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6aac-8e87-41e0-b261-bbf2950faf0c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:57.848988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b6aac-8e87-41e0-b261-bbf2950faf0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:57.860612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b6aac-8e87-41e0-b261-bbf2950faf0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-651b6aac-8e87-41e0-b261-bbf2950faf0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:15:57.871824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:15:57.872839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:15:57.873786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 7/14] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:15:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:15:57.879247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b6aac-8e87-41e0-b261-bbf2950faf0c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:57.922125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ce1e0-e3a5-45f2-82da-96d0e3a49c60 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:57.924836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:15:57.959208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d683521-47fc-47b0-8245-84015c216f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:57.960099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d683521-47fc-47b0-8245-84015c216f93 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:57.960390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d683521-47fc-47b0-8245-84015c216f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:57.960673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d683521-47fc-47b0-8245-84015c216f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:57.963333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:15:57.981319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b6aac-8e87-41e0-b261-bbf2950faf0c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:58.015625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:58.016288 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:58.016570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:58.016848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:58.017633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:15:58.017948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:58.019335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:58.027678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d683521-47fc-47b0-8245-84015c216f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:58.029209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d683521-47fc-47b0-8245-84015c216f93 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:58.031056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 8/15] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:15:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:15:58.036441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:58.126622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 8/16] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:15:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 03:15:58.131768 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d683521-47fc-47b0-8245-84015c216f93 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:58.133544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32db56-fd1c-4db7-a737-14b8fa4653c5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:58.134413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:15:58.167265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36051-978e-4fe6-899a-a95bbbc38c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:58.168144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36051-978e-4fe6-899a-a95bbbc38c2a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:58.168431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36051-978e-4fe6-899a-a95bbbc38c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:58.168711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36051-978e-4fe6-899a-a95bbbc38c2a None admin] PATH_INFO: `/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:58.169947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb36051-978e-4fe6-899a-a95bbbc38c2a None admin] RBAC: Authorizing `identity:get_user(user_id=f454c88a6bba4ab9ba311bbbd1eb0425)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:58.171972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb36051-978e-4fe6-899a-a95bbbc38c2a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:58.173646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 9/17] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:15:58 2022] GET /identity/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:15:58.178901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36051-978e-4fe6-899a-a95bbbc38c2a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:58.192766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:15:58.230433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d683521-47fc-47b0-8245-84015c216f93 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:58.263360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:58.264151 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:58.264440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:58.264722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:58.274507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36051-978e-4fe6-899a-a95bbbc38c2a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:58.283369 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:58.284836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:15:58.305781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:15:58.306687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:15:58.307472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:15:58.307472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:15:58.307472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:15:58.307472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:15:58.307472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 9/18] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:15:58.312774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb304d-de54-4ad0-a285-d1e5617e5fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:58.312774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb304d-de54-4ad0-a285-d1e5617e5fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:58.312774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb304d-de54-4ad0-a285-d1e5617e5fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:58.312774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb304d-de54-4ad0-a285-d1e5617e5fa2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:58.313121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb304d-de54-4ad0-a285-d1e5617e5fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:58.313121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb304d-de54-4ad0-a285-d1e5617e5fa2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:58.313725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:58.313828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 10/19] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:15:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:15:58.319025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb304d-de54-4ad0-a285-d1e5617e5fa2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:15:58.370204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:15:58.397361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:15:58.415922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb304d-de54-4ad0-a285-d1e5617e5fa2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:58.417386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e87276-2842-46b7-827d-5ad02196b26a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:15:58.453024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ceb74-97dc-4f0e-a98f-f1f0e99464df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:58.453704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ceb74-97dc-4f0e-a98f-f1f0e99464df None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:58.453984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ceb74-97dc-4f0e-a98f-f1f0e99464df None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:58.454255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ceb74-97dc-4f0e-a98f-f1f0e99464df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:58.455073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ceb74-97dc-4f0e-a98f-f1f0e99464df None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.domain.id=default, user.id=f454c88a6bba4ab9ba311bbbd1eb0425) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:15:58.455379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ceb74-97dc-4f0e-a98f-f1f0e99464df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:58.456828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ceb74-97dc-4f0e-a98f-f1f0e99464df None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:58.457070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3772099-8abe-458f-b865-535edad9d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:15:58.457753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3772099-8abe-458f-b865-535edad9d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:15:58.458035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3772099-8abe-458f-b865-535edad9d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:15:58.458308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3772099-8abe-458f-b865-535edad9d1c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:15:58.459024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3772099-8abe-458f-b865-535edad9d1c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:15:58.459329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3772099-8abe-458f-b865-535edad9d1c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:15:58.460697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3772099-8abe-458f-b865-535edad9d1c4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:15:58.463530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 11/20] 10.222.0.41 () {56 vars in 1322 bytes} [Sat May 28 03:15:58 2022] GET /identity/v3/role_assignments?user.id=f454c88a6bba4ab9ba311bbbd1eb0425&scope.domain.id=default&role.id=bb0756f8af694633b9743a958eb46e7a => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:15:58.590605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 10/21] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:01.391734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426229-3c55-4562-853f-6764268faf1b None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:01.392735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426229-3c55-4562-853f-6764268faf1b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:01.393259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426229-3c55-4562-853f-6764268faf1b None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:01.394897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 12/22] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:01.400321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201022b0-d4ba-4764-bef5-c711394528db None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:01.401182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201022b0-d4ba-4764-bef5-c711394528db None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:01.401663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201022b0-d4ba-4764-bef5-c711394528db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:01.423698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-201022b0-d4ba-4764-bef5-c711394528db None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:01.425258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-201022b0-d4ba-4764-bef5-c711394528db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:01.520095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 11/23] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:01.558346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc426229-3c55-4562-853f-6764268faf1b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:01.702869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc426229-3c55-4562-853f-6764268faf1b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:01.743947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d376-21af-47c5-affd-2578180781bf None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:01.744579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d376-21af-47c5-affd-2578180781bf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:01.745157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339d376-21af-47c5-affd-2578180781bf None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:01.746775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 12/24] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:01.756995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4867a3-4752-4468-978a-6648004dc475 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:01.757608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4867a3-4752-4468-978a-6648004dc475 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:01.758104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4867a3-4752-4468-978a-6648004dc475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:01.764860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab25fa-6933-4b90-b86e-b4b2a555d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:01.770131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab25fa-6933-4b90-b86e-b4b2a555d2da None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:01.770658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab25fa-6933-4b90-b86e-b4b2a555d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:01.771133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab25fa-6933-4b90-b86e-b4b2a555d2da None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:01.772098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab25fa-6933-4b90-b86e-b4b2a555d2da None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:01.778668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab25fa-6933-4b90-b86e-b4b2a555d2da None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:01.790027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6e4867a3-4752-4468-978a-6648004dc475 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:01.791997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6e4867a3-4752-4468-978a-6648004dc475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:01.865474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:01.905223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 13/25] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:01.907019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db21a8-6b23-4f4d-8a63-fa12173efc5e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:01.907316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db21a8-6b23-4f4d-8a63-fa12173efc5e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:01.907598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db21a8-6b23-4f4d-8a63-fa12173efc5e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:01.909139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 14/26] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:01.914346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454882d-9376-4a86-8272-af3c0be60d97 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:01.914640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454882d-9376-4a86-8272-af3c0be60d97 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:01.914920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454882d-9376-4a86-8272-af3c0be60d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:01.946316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9454882d-9376-4a86-8272-af3c0be60d97 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:01.947507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9454882d-9376-4a86-8272-af3c0be60d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:02.023334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 13/27] 10.222.0.41 () {60 vars in 1131 bytes} [Sat May 28 03:16:01 2022] POST /identity/v3/roles => generated 219 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:02.025049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab25fa-6933-4b90-b86e-b4b2a555d2da None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:02.031210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:02.105247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 15/28] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:02.110693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daff35-f265-4bca-954b-e54f0f1c4133 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.111116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daff35-f265-4bca-954b-e54f0f1c4133 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.111350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daff35-f265-4bca-954b-e54f0f1c4133 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.112850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 16/29] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:02.130837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e5849-5f48-404f-8642-5273a20fdd21 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.131140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e5849-5f48-404f-8642-5273a20fdd21 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.131403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e5849-5f48-404f-8642-5273a20fdd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.149660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab25fa-6933-4b90-b86e-b4b2a555d2da None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:02.158834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-945e5849-5f48-404f-8642-5273a20fdd21 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:02.158834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-945e5849-5f48-404f-8642-5273a20fdd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:02.196604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df6184-0747-43f7-9bf8-743dcad5b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:02.197903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df6184-0747-43f7-9bf8-743dcad5b3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.198377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df6184-0747-43f7-9bf8-743dcad5b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.198835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df6184-0747-43f7-9bf8-743dcad5b3c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.224023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df6184-0747-43f7-9bf8-743dcad5b3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:02.246359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 17/30] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-57df6184-0747-43f7-9bf8-743dcad5b3c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:02.247019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:02.247858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:02.249272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:02.249874 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e5849-5f48-404f-8642-5273a20fdd21 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:02.253226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 14/31] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:02.257378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df6184-0747-43f7-9bf8-743dcad5b3c9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:02.303587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:02.350960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:02.394030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df6184-0747-43f7-9bf8-743dcad5b3c9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:02.400163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e5849-5f48-404f-8642-5273a20fdd21 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:02.441613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:02.442295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.442602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.442884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.443624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:02.443936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:02.445337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccc7e8-240c-4f17-84f3-1dc9a5e0c7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:02.445770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:02.446884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccc7e8-240c-4f17-84f3-1dc9a5e0c7fb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.447246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccc7e8-240c-4f17-84f3-1dc9a5e0c7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.447529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccc7e8-240c-4f17-84f3-1dc9a5e0c7fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.454774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 15/32] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:02.455582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:02.461519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccc7e8-240c-4f17-84f3-1dc9a5e0c7fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:02.463212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccc7e8-240c-4f17-84f3-1dc9a5e0c7fb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6eccc7e8-240c-4f17-84f3-1dc9a5e0c7fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:16:02.483963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:02.484797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:02.485634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:16:02.485634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:16:02.485634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:16:02.485634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:02.485844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 18/33] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:02.491192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f95717-515f-4297-bf1c-ce5fb68470ce None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.491192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f95717-515f-4297-bf1c-ce5fb68470ce None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.491192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f95717-515f-4297-bf1c-ce5fb68470ce None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.491192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 19/34] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:02.493703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f95717-515f-4297-bf1c-ce5fb68470ce None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:02.562907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349d9c1-d064-4c47-943b-904b02bd0223 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:02.594492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:02.595932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:02.606388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556504d-477f-4003-98bd-83ea16d4ca1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:02.607026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556504d-477f-4003-98bd-83ea16d4ca1d None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.616482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556504d-477f-4003-98bd-83ea16d4ca1d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.616482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556504d-477f-4003-98bd-83ea16d4ca1d None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.616482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556504d-477f-4003-98bd-83ea16d4ca1d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:02.616724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f95717-515f-4297-bf1c-ce5fb68470ce None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:02.616839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556504d-477f-4003-98bd-83ea16d4ca1d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:02.643025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaf874-21e0-4939-a286-2209b051aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:02.643896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaf874-21e0-4939-a286-2209b051aa90 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.644181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaf874-21e0-4939-a286-2209b051aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.644457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eaf874-21e0-4939-a286-2209b051aa90 None admin] PATH_INFO: `/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.646114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaf874-21e0-4939-a286-2209b051aa90 None admin] RBAC: Authorizing `identity:get_user(user_id=f454c88a6bba4ab9ba311bbbd1eb0425)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:02.647557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eaf874-21e0-4939-a286-2209b051aa90 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:02.649485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 20/35] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:02 2022] GET /identity/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:02.649985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaf874-21e0-4939-a286-2209b051aa90 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:02.662083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 16/36] 10.222.0.41 () {60 vars in 1131 bytes} [Sat May 28 03:16:02 2022] POST /identity/v3/roles => generated 225 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:02.663406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095161b-5302-4673-88f2-916bd2a69933 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.663698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095161b-5302-4673-88f2-916bd2a69933 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.663969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095161b-5302-4673-88f2-916bd2a69933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.687394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a095161b-5302-4673-88f2-916bd2a69933 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:02.689012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a095161b-5302-4673-88f2-916bd2a69933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:02.766181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 17/37] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:02.770061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095161b-5302-4673-88f2-916bd2a69933 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:02.792501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eaf874-21e0-4939-a286-2209b051aa90 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:02.843824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:02.845077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.845666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.846619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.848015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:02.848696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:02.850633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:02.867589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 21/38] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:02.868889 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:02.902283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:02.918132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095161b-5302-4673-88f2-916bd2a69933 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:02.939301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:02.972256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b425eb8-de24-487a-bafd-3ca8d8e87175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:02.973232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b425eb8-de24-487a-bafd-3ca8d8e87175 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:02.973597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b425eb8-de24-487a-bafd-3ca8d8e87175 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:02.973994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b425eb8-de24-487a-bafd-3ca8d8e87175 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:02.975222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b425eb8-de24-487a-bafd-3ca8d8e87175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:02.976712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b425eb8-de24-487a-bafd-3ca8d8e87175 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:02.981956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 18/39] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:02.981956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b425eb8-de24-487a-bafd-3ca8d8e87175 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:03.013567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357008c-2a7a-46f7-b088-845f66327797 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:03.049617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b91a86-3153-4428-baf2-3d0fbcd926f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:03.050290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b91a86-3153-4428-baf2-3d0fbcd926f7 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:03.050581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b91a86-3153-4428-baf2-3d0fbcd926f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:03.050864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b91a86-3153-4428-baf2-3d0fbcd926f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:03.051612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b91a86-3153-4428-baf2-3d0fbcd926f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:03.053011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b91a86-3153-4428-baf2-3d0fbcd926f7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:03.084685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b425eb8-de24-487a-bafd-3ca8d8e87175 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:03.117603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41661b0d-e8e6-4a1d-a6e5-f7509690a96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:03.118202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41661b0d-e8e6-4a1d-a6e5-f7509690a96d None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:03.118493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41661b0d-e8e6-4a1d-a6e5-f7509690a96d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:03.118778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41661b0d-e8e6-4a1d-a6e5-f7509690a96d None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:03.119485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41661b0d-e8e6-4a1d-a6e5-f7509690a96d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:03.120944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41661b0d-e8e6-4a1d-a6e5-f7509690a96d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:03.215699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 19/40] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:16:02 2022] POST /identity/v3/projects => generated 301 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:03.216502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41661b0d-e8e6-4a1d-a6e5-f7509690a96d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:03.230223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 22/41] 10.222.0.41 () {60 vars in 1131 bytes} [Sat May 28 03:16:02 2022] POST /identity/v3/roles => generated 223 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:03.231836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511efc-e2f9-4151-8ce8-c8beeee9573a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:03.232132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511efc-e2f9-4151-8ce8-c8beeee9573a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:03.232411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50511efc-e2f9-4151-8ce8-c8beeee9573a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:03.233900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 23/42] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:03.247885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f8c0f-486f-44a7-909e-c55d077093ba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:03.248331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f8c0f-486f-44a7-909e-c55d077093ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:03.248711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f8c0f-486f-44a7-909e-c55d077093ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:03.270022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:03.277252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-689f8c0f-486f-44a7-909e-c55d077093ba None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:03.279190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-689f8c0f-486f-44a7-909e-c55d077093ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:03.350235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41661b0d-e8e6-4a1d-a6e5-f7509690a96d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:03.363401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 24/43] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:03.388519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f8c0f-486f-44a7-909e-c55d077093ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:03.392017 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652253f2-825f-4455-8fb9-7d2758f16489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:03.393270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652253f2-825f-4455-8fb9-7d2758f16489 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:03.393751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652253f2-825f-4455-8fb9-7d2758f16489 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:03.394255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652253f2-825f-4455-8fb9-7d2758f16489 None admin] PATH_INFO: `/v3/domains/default/users/f454c88a6bba4ab9ba311bbbd1eb0425/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:03.397310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652253f2-825f-4455-8fb9-7d2758f16489 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f454c88a6bba4ab9ba311bbbd1eb0425, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:03.399168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652253f2-825f-4455-8fb9-7d2758f16489 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:03.401493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:03.497972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f8c0f-486f-44a7-909e-c55d077093ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:03.516363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 20/44] 10.222.0.41 () {58 vars in 1361 bytes} [Sat May 28 03:16:03 2022] PUT /identity/v3/domains/default/users/f454c88a6bba4ab9ba311bbbd1eb0425/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:03.535211 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444627e-3357-4d49-8f46-0a664a8eaa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:03.536050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444627e-3357-4d49-8f46-0a664a8eaa0a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:03.536345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444627e-3357-4d49-8f46-0a664a8eaa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:03.536630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444627e-3357-4d49-8f46-0a664a8eaa0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:03.537962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444627e-3357-4d49-8f46-0a664a8eaa0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:03.539656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444627e-3357-4d49-8f46-0a664a8eaa0a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:03.546459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 25/45] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:03.547841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652253f2-825f-4455-8fb9-7d2758f16489 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:03.656791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:03.662380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652253f2-825f-4455-8fb9-7d2758f16489 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:03.701317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbcd12-4b46-40eb-ae8e-edef525e2d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:03.701995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbcd12-4b46-40eb-ae8e-edef525e2d43 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:03.703012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbcd12-4b46-40eb-ae8e-edef525e2d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:03.703523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbcd12-4b46-40eb-ae8e-edef525e2d43 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:03.704634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbcd12-4b46-40eb-ae8e-edef525e2d43 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:03.706318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbcd12-4b46-40eb-ae8e-edef525e2d43 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:03.787525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 21/46] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:16:03 2022] POST /identity/v3/projects => generated 312 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:05.700335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6610bcf-7956-4fd3-99bf-74cceca74b99 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:05.700881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6610bcf-7956-4fd3-99bf-74cceca74b99 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:05.700881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6610bcf-7956-4fd3-99bf-74cceca74b99 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:05.702926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 26/47] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:05.708147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3265c4-2b03-4050-aa78-d1ae4985edde None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:05.708546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3265c4-2b03-4050-aa78-d1ae4985edde None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:05.708862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3265c4-2b03-4050-aa78-d1ae4985edde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:05.725564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8d3265c4-2b03-4050-aa78-d1ae4985edde None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:05.726754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8d3265c4-2b03-4050-aa78-d1ae4985edde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:05.794721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 22/48] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:05.818184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6610bcf-7956-4fd3-99bf-74cceca74b99 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:05.896111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:05.923399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6610bcf-7956-4fd3-99bf-74cceca74b99 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:05.958128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae7586-ea91-4bbe-8a99-329d9c60f753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:05.958841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7586-ea91-4bbe-8a99-329d9c60f753 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:05.959134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7586-ea91-4bbe-8a99-329d9c60f753 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:05.959421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7586-ea91-4bbe-8a99-329d9c60f753 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:05.970190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696fd16-de45-457a-961f-3d480a813342 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:05.970489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696fd16-de45-457a-961f-3d480a813342 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:05.970724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae7586-ea91-4bbe-8a99-329d9c60f753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:05.970817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696fd16-de45-457a-961f-3d480a813342 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:05.972432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 23/49] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:05.977393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd5fb2-b690-4b8a-9cab-b4788dbc4766 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:05.977759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd5fb2-b690-4b8a-9cab-b4788dbc4766 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:05.978044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd5fb2-b690-4b8a-9cab-b4788dbc4766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-cbae7586-ea91-4bbe-8a99-329d9c60f753 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:05.979951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:05.980832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:05.981634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 27/50] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:05.986947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae7586-ea91-4bbe-8a99-329d9c60f753 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:05.994929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-70fd5fb2-b690-4b8a-9cab-b4788dbc4766 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:05.996117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-70fd5fb2-b690-4b8a-9cab-b4788dbc4766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:06.063496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 24/51] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:06.085816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae7586-ea91-4bbe-8a99-329d9c60f753 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:06.086235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd5fb2-b690-4b8a-9cab-b4788dbc4766 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:06.097045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:06.121649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:06.122322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:06.128090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:06.128090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:06.128090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:06.128090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:06.128090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:06.128090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:06.138297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 28/52] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:06.143839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:06.186209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd5fb2-b690-4b8a-9cab-b4788dbc4766 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:06.218777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592616a4-0801-42d6-b242-c062be63cb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:06.219519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592616a4-0801-42d6-b242-c062be63cb0a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:06.219806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592616a4-0801-42d6-b242-c062be63cb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:06.220089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592616a4-0801-42d6-b242-c062be63cb0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:06.221565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592616a4-0801-42d6-b242-c062be63cb0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:06.223103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592616a4-0801-42d6-b242-c062be63cb0a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:06.224723 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 25/53] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:06.230303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592616a4-0801-42d6-b242-c062be63cb0a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:06.241191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da007e9-46a7-45f7-b941-1f7bbfbdb01b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:06.276031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2714e0-ca0b-436f-bb5d-7675e71f3fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:06.276901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2714e0-ca0b-436f-bb5d-7675e71f3fcb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:06.277226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2714e0-ca0b-436f-bb5d-7675e71f3fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:06.277515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2714e0-ca0b-436f-bb5d-7675e71f3fcb None admin] PATH_INFO: `/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:06.278718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2714e0-ca0b-436f-bb5d-7675e71f3fcb None admin] RBAC: Authorizing `identity:get_user(user_id=f454c88a6bba4ab9ba311bbbd1eb0425)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:06.280175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2714e0-ca0b-436f-bb5d-7675e71f3fcb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:06.281862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 29/54] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:06 2022] GET /identity/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:06.287148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2714e0-ca0b-436f-bb5d-7675e71f3fcb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:06.305773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:06.328431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592616a4-0801-42d6-b242-c062be63cb0a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:06.348124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:06.361968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea145af1-3415-4a9e-a965-e0062dccf696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:06.362634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea145af1-3415-4a9e-a965-e0062dccf696 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:06.362921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea145af1-3415-4a9e-a965-e0062dccf696 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:06.363203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea145af1-3415-4a9e-a965-e0062dccf696 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:06.363909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea145af1-3415-4a9e-a965-e0062dccf696 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:06.365386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea145af1-3415-4a9e-a965-e0062dccf696 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:06.385728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2714e0-ca0b-436f-bb5d-7675e71f3fcb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:06.419409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6e663-3e53-499c-b401-dccef6b7e687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:06.420192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6e663-3e53-499c-b401-dccef6b7e687 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:06.424921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6e663-3e53-499c-b401-dccef6b7e687 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:06.424921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6e663-3e53-499c-b401-dccef6b7e687 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:06.424921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6e663-3e53-499c-b401-dccef6b7e687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:06.424921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac6e663-3e53-499c-b401-dccef6b7e687 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:06.425341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 30/55] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:06.430825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6e663-3e53-499c-b401-dccef6b7e687 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:06.449733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 26/56] 10.222.0.41 () {60 vars in 1140 bytes} [Sat May 28 03:16:06 2022] POST /identity/v3/projects => generated 298 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:16:06.541220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac6e663-3e53-499c-b401-dccef6b7e687 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:06.572698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:06.578478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717a239-a62d-47c4-9a6e-cccf2f36a6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:06.579157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717a239-a62d-47c4-9a6e-cccf2f36a6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:06.579448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717a239-a62d-47c4-9a6e-cccf2f36a6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:06.579730 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717a239-a62d-47c4-9a6e-cccf2f36a6e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:06.585322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717a239-a62d-47c4-9a6e-cccf2f36a6e4 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.domain.id=default, user.id=f454c88a6bba4ab9ba311bbbd1eb0425) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:06.585322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717a239-a62d-47c4-9a6e-cccf2f36a6e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:06.585322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5717a239-a62d-47c4-9a6e-cccf2f36a6e4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:06.592181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 31/57] 10.222.0.41 () {56 vars in 1322 bytes} [Sat May 28 03:16:06 2022] GET /identity/v3/role_assignments?user.id=f454c88a6bba4ab9ba311bbbd1eb0425&scope.domain.id=default&role.id=bb0756f8af694633b9743a958eb46e7a => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:08.637042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c4248-b91e-409c-b6b6-5d9ad3600d68 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:08.637575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c4248-b91e-409c-b6b6-5d9ad3600d68 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:08.637656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c4248-b91e-409c-b6b6-5d9ad3600d68 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:08.638995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 27/58] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:08.644254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d536b1-b8e0-492d-964a-718609acf00f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:08.644674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d536b1-b8e0-492d-964a-718609acf00f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:08.644821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d536b1-b8e0-492d-964a-718609acf00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:08.661775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-80d536b1-b8e0-492d-964a-718609acf00f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:08.662943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-80d536b1-b8e0-492d-964a-718609acf00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:08.726553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 32/59] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:08.751131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c4248-b91e-409c-b6b6-5d9ad3600d68 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:08.848989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c4248-b91e-409c-b6b6-5d9ad3600d68 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:08.873880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:08.881692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca969a-0c9b-45a6-b8f6-ea4a67c3bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:08.882515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca969a-0c9b-45a6-b8f6-ea4a67c3bde5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:08.893273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca969a-0c9b-45a6-b8f6-ea4a67c3bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:08.893273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca969a-0c9b-45a6-b8f6-ea4a67c3bde5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:08.893273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca969a-0c9b-45a6-b8f6-ea4a67c3bde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:08.893273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca969a-0c9b-45a6-b8f6-ea4a67c3bde5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:08.893273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 28/60] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:08.893715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d536b1-b8e0-492d-964a-718609acf00f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:08.989834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d536b1-b8e0-492d-964a-718609acf00f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:09.013866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:09.022030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4556e-cc32-4e5a-87a8-2de7ea9834b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:09.022703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4556e-cc32-4e5a-87a8-2de7ea9834b7 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:09.023023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4556e-cc32-4e5a-87a8-2de7ea9834b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:09.023336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4556e-cc32-4e5a-87a8-2de7ea9834b7 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:09.024055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4556e-cc32-4e5a-87a8-2de7ea9834b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:09.025541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4556e-cc32-4e5a-87a8-2de7ea9834b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:09.116811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 33/61] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:16:08 2022] POST /identity/v3/users => generated 275 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:12.424084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eefed3-9de2-4d1f-a8bf-afe0b014f236 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.425072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eefed3-9de2-4d1f-a8bf-afe0b014f236 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.425618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eefed3-9de2-4d1f-a8bf-afe0b014f236 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.427202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 29/62] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:12.432578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2198e0-c0c2-46b9-93d8-e4ecff350354 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.433473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2198e0-c0c2-46b9-93d8-e4ecff350354 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.433954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2198e0-c0c2-46b9-93d8-e4ecff350354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.457987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1b2198e0-c0c2-46b9-93d8-e4ecff350354 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:12.459562 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b2198e0-c0c2-46b9-93d8-e4ecff350354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:12.500111 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746d75e-338e-470f-8abb-d8c94a81f144 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.500718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746d75e-338e-470f-8abb-d8c94a81f144 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.501237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746d75e-338e-470f-8abb-d8c94a81f144 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.502787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 30/63] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:12.507456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca986e-8755-429c-ac5d-aaaaccba8fd1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.507976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca986e-8755-429c-ac5d-aaaaccba8fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.508475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca986e-8755-429c-ac5d-aaaaccba8fd1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.509955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 31/64] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:12.511244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a4aac-bb28-4e01-b29d-9578ef575c68 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.511746 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a4aac-bb28-4e01-b29d-9578ef575c68 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.512278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a4aac-bb28-4e01-b29d-9578ef575c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.534344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-923a4aac-bb28-4e01-b29d-9578ef575c68 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:12.536074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-923a4aac-bb28-4e01-b29d-9578ef575c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:12.547821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 34/65] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:12.557326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a0367-598e-453c-9474-2efc6c0c1cda None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.557657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a0367-598e-453c-9474-2efc6c0c1cda None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.557954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a0367-598e-453c-9474-2efc6c0c1cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.589742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7f6a0367-598e-453c-9474-2efc6c0c1cda None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:12.590958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7f6a0367-598e-453c-9474-2efc6c0c1cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:12.613512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 32/66] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:12.614630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a4aac-bb28-4e01-b29d-9578ef575c68 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:12.689216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 35/67] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:12.689216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a0367-598e-453c-9474-2efc6c0c1cda None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:12.743543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:12.750810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a4aac-bb28-4e01-b29d-9578ef575c68 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:12.779907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:12.789256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce69d15-4a30-489d-a9fb-4a516154a318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:12.790263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce69d15-4a30-489d-a9fb-4a516154a318 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.790688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce69d15-4a30-489d-a9fb-4a516154a318 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.791102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce69d15-4a30-489d-a9fb-4a516154a318 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.806085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce69d15-4a30-489d-a9fb-4a516154a318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:12.812469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a0367-598e-453c-9474-2efc6c0c1cda None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9ce69d15-4a30-489d-a9fb-4a516154a318 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:12.815854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:12.816774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:12.817573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:12.820013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 33/68] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:12.821069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce69d15-4a30-489d-a9fb-4a516154a318 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:12.850032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdadb-6c55-4788-a24b-684d9ca21bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:12.851038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdadb-6c55-4788-a24b-684d9ca21bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.851295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdadb-6c55-4788-a24b-684d9ca21bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.851640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdadb-6c55-4788-a24b-684d9ca21bb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.862891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fdadb-6c55-4788-a24b-684d9ca21bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6b3fdadb-6c55-4788-a24b-684d9ca21bb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:12.870757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:12.871604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:12.872432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 36/69] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:12.873036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdadb-6c55-4788-a24b-684d9ca21bb1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:12.943141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce69d15-4a30-489d-a9fb-4a516154a318 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:12.978103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd739c52-c09a-4151-947b-7218025999e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:12.978887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd739c52-c09a-4151-947b-7218025999e0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:12.979340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd739c52-c09a-4151-947b-7218025999e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:12.979622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd739c52-c09a-4151-947b-7218025999e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:12.990750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd739c52-c09a-4151-947b-7218025999e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:12.997840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdadb-6c55-4788-a24b-684d9ca21bb1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-fd739c52-c09a-4151-947b-7218025999e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:12.998929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:13.000257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:13.000999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 34/70] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:13.001656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd739c52-c09a-4151-947b-7218025999e0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.002652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.038961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a351ad-ca3b-46b8-9038-74d913a58693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.039766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a351ad-ca3b-46b8-9038-74d913a58693 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.040128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a351ad-ca3b-46b8-9038-74d913a58693 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.040481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a351ad-ca3b-46b8-9038-74d913a58693 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.041442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a351ad-ca3b-46b8-9038-74d913a58693 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:13.041762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a351ad-ca3b-46b8-9038-74d913a58693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:13.043170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a351ad-ca3b-46b8-9038-74d913a58693 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:13.051952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 37/71] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:13.053409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c81c1f-aba7-4fe6-8361-11f5fb94c07a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.053835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c81c1f-aba7-4fe6-8361-11f5fb94c07a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.054248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c81c1f-aba7-4fe6-8361-11f5fb94c07a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.055609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 38/72] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:13.056717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c81c1f-aba7-4fe6-8361-11f5fb94c07a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.057523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.118678 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd739c52-c09a-4151-947b-7218025999e0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:13.157646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.158324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.158614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.158897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.159770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:13.160091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:13.161520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:13.170649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 35/73] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:13.171784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.174886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c81c1f-aba7-4fe6-8361-11f5fb94c07a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:13.211399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bee83-397f-490a-b8e5-fd29e2eeb2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.211979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bee83-397f-490a-b8e5-fd29e2eeb2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.212266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bee83-397f-490a-b8e5-fd29e2eeb2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.212543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bee83-397f-490a-b8e5-fd29e2eeb2a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.213315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bee83-397f-490a-b8e5-fd29e2eeb2a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:13.213625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bee83-397f-490a-b8e5-fd29e2eeb2a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:13.214988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bee83-397f-490a-b8e5-fd29e2eeb2a2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:13.225167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 39/74] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:13.226590 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b43111-9c91-42e3-875f-1e0e0197d13d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.226884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b43111-9c91-42e3-875f-1e0e0197d13d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.227256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b43111-9c91-42e3-875f-1e0e0197d13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.245486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-50b43111-9c91-42e3-875f-1e0e0197d13d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:13.246925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-50b43111-9c91-42e3-875f-1e0e0197d13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:13.268943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.284766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92153-5aa3-42c7-a83a-ca9abd6a9088 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:13.294662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.317905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 40/75] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:13.318826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b43111-9c91-42e3-875f-1e0e0197d13d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.320206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51be6-4256-47c4-83a1-3fe121a7efd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.321485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51be6-4256-47c4-83a1-3fe121a7efd0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.321981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51be6-4256-47c4-83a1-3fe121a7efd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.322353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb51be6-4256-47c4-83a1-3fe121a7efd0 None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.336387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51be6-4256-47c4-83a1-3fe121a7efd0 None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:13.338145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb51be6-4256-47c4-83a1-3fe121a7efd0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:13.339992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 36/76] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:13.340810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51be6-4256-47c4-83a1-3fe121a7efd0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.417045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b43111-9c91-42e3-875f-1e0e0197d13d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:13.439328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb51be6-4256-47c4-83a1-3fe121a7efd0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:13.452593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c87ea-9390-44a7-a65e-4400a77a45a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.453503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c87ea-9390-44a7-a65e-4400a77a45a9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.453791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c87ea-9390-44a7-a65e-4400a77a45a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.454080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c87ea-9390-44a7-a65e-4400a77a45a9 None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.455280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c87ea-9390-44a7-a65e-4400a77a45a9 None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:13.456740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c87ea-9390-44a7-a65e-4400a77a45a9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:13.458397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 41/77] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:13.459560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8eaac-bf7d-4952-8cc8-8959e55124fa None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.459857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8eaac-bf7d-4952-8cc8-8959e55124fa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.460140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8eaac-bf7d-4952-8cc8-8959e55124fa None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.462467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 42/78] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:13.462467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8eaac-bf7d-4952-8cc8-8959e55124fa None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.476568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0e0ae-dff2-4f41-a2d5-204cbb5b736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.477486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0e0ae-dff2-4f41-a2d5-204cbb5b736f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.477772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0e0ae-dff2-4f41-a2d5-204cbb5b736f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.478048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0e0ae-dff2-4f41-a2d5-204cbb5b736f None admin] PATH_INFO: `/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.479244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0e0ae-dff2-4f41-a2d5-204cbb5b736f None admin] RBAC: Authorizing `identity:get_user(user_id=f454c88a6bba4ab9ba311bbbd1eb0425)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:13.480713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0e0ae-dff2-4f41-a2d5-204cbb5b736f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:13.482344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 37/79] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:13.483096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0e0ae-dff2-4f41-a2d5-204cbb5b736f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.491208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.516300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.567845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8eaac-bf7d-4952-8cc8-8959e55124fa None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:13.580854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0e0ae-dff2-4f41-a2d5-204cbb5b736f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:13.602147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548b66a-67a5-492d-9c7c-61b4b4165333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.602939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548b66a-67a5-492d-9c7c-61b4b4165333 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.603230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548b66a-67a5-492d-9c7c-61b4b4165333 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.603509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548b66a-67a5-492d-9c7c-61b4b4165333 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.613799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548b66a-67a5-492d-9c7c-61b4b4165333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:13.613997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef79e4f-56b2-4280-a3d0-772c5f8b7a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.614570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef79e4f-56b2-4280-a3d0-772c5f8b7a8a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.614857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef79e4f-56b2-4280-a3d0-772c5f8b7a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.615135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef79e4f-56b2-4280-a3d0-772c5f8b7a8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b548b66a-67a5-492d-9c7c-61b4b4165333 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:13.621513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:13.622339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:13.623067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 43/80] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:13.623678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548b66a-67a5-492d-9c7c-61b4b4165333 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.684903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef79e4f-56b2-4280-a3d0-772c5f8b7a8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:13.686428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef79e4f-56b2-4280-a3d0-772c5f8b7a8a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:13.706130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.721198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.722452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548b66a-67a5-492d-9c7c-61b4b4165333 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:13.754909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2dc1f-28ed-4f66-aac6-458d8ec75321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:13.755555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dc1f-28ed-4f66-aac6-458d8ec75321 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.755864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dc1f-28ed-4f66-aac6-458d8ec75321 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.756128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dc1f-28ed-4f66-aac6-458d8ec75321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.781684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 38/81] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:13.783020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac59d07-6c7c-4829-814e-cf8d0ee8013e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:13.783318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac59d07-6c7c-4829-814e-cf8d0ee8013e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:13.783658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac59d07-6c7c-4829-814e-cf8d0ee8013e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:13.800587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2ac59d07-6c7c-4829-814e-cf8d0ee8013e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:13.801810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2ac59d07-6c7c-4829-814e-cf8d0ee8013e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:13.831522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2dc1f-28ed-4f66-aac6-458d8ec75321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:13.832984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2dc1f-28ed-4f66-aac6-458d8ec75321 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:13.870100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 39/82] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:13.881289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac59d07-6c7c-4829-814e-cf8d0ee8013e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.926529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.932677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:13.943037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 44/83] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:13.943668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2dc1f-28ed-4f66-aac6-458d8ec75321 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:13.974512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac59d07-6c7c-4829-814e-cf8d0ee8013e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.013469 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be41049-ca21-4591-97f5-7738fa58ad37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.014127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be41049-ca21-4591-97f5-7738fa58ad37 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.014414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be41049-ca21-4591-97f5-7738fa58ad37 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.015245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be41049-ca21-4591-97f5-7738fa58ad37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.046335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2dc1f-28ed-4f66-aac6-458d8ec75321 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.080235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.080891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.081210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.081489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.081942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be41049-ca21-4591-97f5-7738fa58ad37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.082226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:14.082533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.083429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be41049-ca21-4591-97f5-7738fa58ad37 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.083895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.091854 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 45/84] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:14.092656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.137249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:14.147375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:14.183536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 40/85] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:14.184363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be41049-ca21-4591-97f5-7738fa58ad37 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.190625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a22bc6-4488-4760-9734-60ece8644887 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.224123 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59022451-5ef4-4fd5-863d-48fae238a05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.225022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59022451-5ef4-4fd5-863d-48fae238a05d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.225292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59022451-5ef4-4fd5-863d-48fae238a05d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.225572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59022451-5ef4-4fd5-863d-48fae238a05d None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.226760 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59022451-5ef4-4fd5-863d-48fae238a05d None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.228244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59022451-5ef4-4fd5-863d-48fae238a05d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.230104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 46/86] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:14.230715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59022451-5ef4-4fd5-863d-48fae238a05d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.281444 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be41049-ca21-4591-97f5-7738fa58ad37 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.314797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f3ad2-56cf-458b-b95c-1b5154a30589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.315708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f3ad2-56cf-458b-b95c-1b5154a30589 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.315980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f3ad2-56cf-458b-b95c-1b5154a30589 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.316252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f3ad2-56cf-458b-b95c-1b5154a30589 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.317437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f3ad2-56cf-458b-b95c-1b5154a30589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.318903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f3ad2-56cf-458b-b95c-1b5154a30589 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.320503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 41/87] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:14.321325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f3ad2-56cf-458b-b95c-1b5154a30589 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.330505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59022451-5ef4-4fd5-863d-48fae238a05d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.345207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:14.354471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:14.366112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d59f2-95b5-425c-bb10-78e72a74c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.366975 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d59f2-95b5-425c-bb10-78e72a74c977 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.367262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d59f2-95b5-425c-bb10-78e72a74c977 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.367543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d59f2-95b5-425c-bb10-78e72a74c977 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.368715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d59f2-95b5-425c-bb10-78e72a74c977 None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.370187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d59f2-95b5-425c-bb10-78e72a74c977 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.371875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 47/88] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:14.372631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d59f2-95b5-425c-bb10-78e72a74c977 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.423269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f3ad2-56cf-458b-b95c-1b5154a30589 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.456083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc23ed4-36be-42b8-a75b-87a01c092e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.456956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23ed4-36be-42b8-a75b-87a01c092e7b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.457275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23ed4-36be-42b8-a75b-87a01c092e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.457556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23ed4-36be-42b8-a75b-87a01c092e7b None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.458754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc23ed4-36be-42b8-a75b-87a01c092e7b None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.460212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc23ed4-36be-42b8-a75b-87a01c092e7b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.461874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 42/89] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:14.462631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc23ed4-36be-42b8-a75b-87a01c092e7b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.475564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d59f2-95b5-425c-bb10-78e72a74c977 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.508830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ce3bb-cc27-4d40-a757-8a97709f3d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.509713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ce3bb-cc27-4d40-a757-8a97709f3d0d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.509996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ce3bb-cc27-4d40-a757-8a97709f3d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.510278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ce3bb-cc27-4d40-a757-8a97709f3d0d None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.511446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ce3bb-cc27-4d40-a757-8a97709f3d0d None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.512885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ce3bb-cc27-4d40-a757-8a97709f3d0d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.514618 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 48/90] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:14.515354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ce3bb-cc27-4d40-a757-8a97709f3d0d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.556840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:14.559731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc23ed4-36be-42b8-a75b-87a01c092e7b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.566201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:14.592782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.593561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.593846 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.594122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.594949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] RBAC: Adding query filter params (role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=cd271be4aaeb4efab5ef86157a14122b, user.id=9a28cf8a3ada43b49924f24ae6da72e0) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:14.595263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.596970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.602235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 43/91] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/role_assignments?user.id=9a28cf8a3ada43b49924f24ae6da72e0&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:14.603152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.616100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ce3bb-cc27-4d40-a757-8a97709f3d0d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.657916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46834f2-51be-4a67-b0ef-618a20ac6894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.658613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46834f2-51be-4a67-b0ef-618a20ac6894 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.658905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46834f2-51be-4a67-b0ef-618a20ac6894 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.659188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46834f2-51be-4a67-b0ef-618a20ac6894 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.659926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46834f2-51be-4a67-b0ef-618a20ac6894 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.661483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46834f2-51be-4a67-b0ef-618a20ac6894 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.717084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06452da-507b-42b9-b881-9f0bf8a6ce0d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.747036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 49/92] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:16:14 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:14.747036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46834f2-51be-4a67-b0ef-618a20ac6894 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.766388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.767348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.767754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.768044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.768979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] RBAC: Adding query filter params (role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=cd271be4aaeb4efab5ef86157a14122b, user.id=9a28cf8a3ada43b49924f24ae6da72e0) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:14.769360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:14.771032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:14.778046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 44/93] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/role_assignments?user.id=9a28cf8a3ada43b49924f24ae6da72e0&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:14.779173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:14.817672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:14.904365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46834f2-51be-4a67-b0ef-618a20ac6894 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:14.917289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:14.958051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbd6cb-6927-4375-b383-e9b8e693d8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:14.958741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbd6cb-6927-4375-b383-e9b8e693d8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:14.959033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbd6cb-6927-4375-b383-e9b8e693d8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:14.959332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbd6cb-6927-4375-b383-e9b8e693d8a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:14.963199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a70b8e-a1e4-44c1-a565-a0878161432e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:15.040921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618e730-5480-4dba-922f-a18067018777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:15.042012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e730-5480-4dba-922f-a18067018777 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:15.042497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e730-5480-4dba-922f-a18067018777 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:15.042965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618e730-5480-4dba-922f-a18067018777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:15.044027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618e730-5480-4dba-922f-a18067018777 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=cd271be4aaeb4efab5ef86157a14122b, user.id=f454c88a6bba4ab9ba311bbbd1eb0425) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:15.044574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618e730-5480-4dba-922f-a18067018777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:15.046920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618e730-5480-4dba-922f-a18067018777 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:15.054759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 45/94] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/role_assignments?user.id=f454c88a6bba4ab9ba311bbbd1eb0425&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=bb0756f8af694633b9743a958eb46e7a => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:15.075981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbd6cb-6927-4375-b383-e9b8e693d8a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:15.078559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbd6cb-6927-4375-b383-e9b8e693d8a4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:15.195593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 50/95] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:15.205603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618e730-5480-4dba-922f-a18067018777 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:15.310115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:15.319820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618e730-5480-4dba-922f-a18067018777 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:15.358165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507ea96-2a1c-490d-b380-ae736835a58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:15.359135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507ea96-2a1c-490d-b380-ae736835a58d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:15.359425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507ea96-2a1c-490d-b380-ae736835a58d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:15.359710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507ea96-2a1c-490d-b380-ae736835a58d None admin] PATH_INFO: `/v3/projects/e75edf2a51ec4bf68976b61eb3e42ab6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:15.360930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507ea96-2a1c-490d-b380-ae736835a58d None admin] RBAC: Authorizing `identity:get_project(project_id=e75edf2a51ec4bf68976b61eb3e42ab6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:15.362502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507ea96-2a1c-490d-b380-ae736835a58d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:15.364219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 46/96] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:15 2022] GET /identity/v3/projects/e75edf2a51ec4bf68976b61eb3e42ab6 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:15.370464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbd6cb-6927-4375-b383-e9b8e693d8a4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:15.381449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:15.479213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbd6cb-6927-4375-b383-e9b8e693d8a4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:15.519846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bde13-dcb2-4ac5-8342-2a1fc99b8108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:15.520517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bde13-dcb2-4ac5-8342-2a1fc99b8108 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:15.520810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bde13-dcb2-4ac5-8342-2a1fc99b8108 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:15.521095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bde13-dcb2-4ac5-8342-2a1fc99b8108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:15.521966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bde13-dcb2-4ac5-8342-2a1fc99b8108 None admin] RBAC: Adding query filter params (role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=e75edf2a51ec4bf68976b61eb3e42ab6, user.id=9a28cf8a3ada43b49924f24ae6da72e0) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:15.522287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bde13-dcb2-4ac5-8342-2a1fc99b8108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:15.523760 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bde13-dcb2-4ac5-8342-2a1fc99b8108 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:15.530108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 51/97] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:15 2022] GET /identity/v3/role_assignments?user.id=9a28cf8a3ada43b49924f24ae6da72e0&scope.project.id=e75edf2a51ec4bf68976b61eb3e42ab6&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:17.089152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08add3-22c4-48e1-8144-3a4bcd3d113b None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:17.089682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08add3-22c4-48e1-8144-3a4bcd3d113b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:17.089756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08add3-22c4-48e1-8144-3a4bcd3d113b None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:17.091093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 47/98] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:17.097368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922486d-9243-4b71-902b-0db54d2e1138 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:17.097750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922486d-9243-4b71-902b-0db54d2e1138 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:17.097946 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922486d-9243-4b71-902b-0db54d2e1138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:17.122708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c922486d-9243-4b71-902b-0db54d2e1138 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:17.123951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c922486d-9243-4b71-902b-0db54d2e1138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:17.206658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:17.211419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 52/99] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:17.263078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08add3-22c4-48e1-8144-3a4bcd3d113b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:17.403280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08add3-22c4-48e1-8144-3a4bcd3d113b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:17.440976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fd431-4fe3-4d65-ad39-160e9fe7fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:17.441845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fd431-4fe3-4d65-ad39-160e9fe7fb98 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:17.442137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fd431-4fe3-4d65-ad39-160e9fe7fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:17.442421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fd431-4fe3-4d65-ad39-160e9fe7fb98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:17.454417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fd431-4fe3-4d65-ad39-160e9fe7fb98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3d1fd431-4fe3-4d65-ad39-160e9fe7fb98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:17.462800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:17.463631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:17.464369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 48/100] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:17.471788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922486d-9243-4b71-902b-0db54d2e1138 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:17.551143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa03fc-fe04-47d4-bdc8-6625a0e898d4 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:17.551779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa03fc-fe04-47d4-bdc8-6625a0e898d4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:17.552265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa03fc-fe04-47d4-bdc8-6625a0e898d4 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:17.553872 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 49/101] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:17.559208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91331bd5-00e6-4e59-b163-36e8679c900a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:17.559757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91331bd5-00e6-4e59-b163-36e8679c900a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:17.560240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91331bd5-00e6-4e59-b163-36e8679c900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:17.589390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-91331bd5-00e6-4e59-b163-36e8679c900a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:17.590951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-91331bd5-00e6-4e59-b163-36e8679c900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:17.599498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922486d-9243-4b71-902b-0db54d2e1138 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:17.625563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:17.646561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:17.653617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:17.654124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:17.654601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:17.656160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:17.656735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:17.660701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:17.674888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 53/102] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:17.681125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:17.751108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 50/103] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:17.777478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:17.792894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91331bd5-00e6-4e59-b163-36e8679c900a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:17.856864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0189f66-0224-4587-9fda-a67f2d46a181 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:17.897866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91331bd5-00e6-4e59-b163-36e8679c900a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:17.934463 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fea1b-f56b-4dd1-b7af-72c6a2c3b7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:17.935533 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fea1b-f56b-4dd1-b7af-72c6a2c3b7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:17.935969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fea1b-f56b-4dd1-b7af-72c6a2c3b7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:17.936921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1fea1b-f56b-4dd1-b7af-72c6a2c3b7c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:17.948421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1fea1b-f56b-4dd1-b7af-72c6a2c3b7c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9d1fea1b-f56b-4dd1-b7af-72c6a2c3b7c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:17.957129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:17.958043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:17.958904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:17.961034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 51/104] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:16:17.978532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e33ad1-cb26-4c58-8012-9b64a27f286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:17.979451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33ad1-cb26-4c58-8012-9b64a27f286a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:17.979609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fea1b-f56b-4dd1-b7af-72c6a2c3b7c9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:17.980028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33ad1-cb26-4c58-8012-9b64a27f286a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:17.980456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e33ad1-cb26-4c58-8012-9b64a27f286a None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:17.985972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e33ad1-cb26-4c58-8012-9b64a27f286a None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:17.987676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e33ad1-cb26-4c58-8012-9b64a27f286a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:17.989959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 54/105] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:17 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:17.995398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e33ad1-cb26-4c58-8012-9b64a27f286a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:18.003914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:18.153402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1fea1b-f56b-4dd1-b7af-72c6a2c3b7c9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:18.194664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:18.195610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.196047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.196475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.197416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:18.197873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:18.199436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:18.208406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 52/106] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:18.228861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:18.239313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e33ad1-cb26-4c58-8012-9b64a27f286a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:18.273664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:18.287100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bc00-83e1-4a41-acaf-0a4998af0228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:18.288549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bc00-83e1-4a41-acaf-0a4998af0228 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.289069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bc00-83e1-4a41-acaf-0a4998af0228 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.289613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bc00-83e1-4a41-acaf-0a4998af0228 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.298549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bc00-83e1-4a41-acaf-0a4998af0228 None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:18.300579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bc00-83e1-4a41-acaf-0a4998af0228 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:18.302756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 55/107] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:17 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:18.323378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bc00-83e1-4a41-acaf-0a4998af0228 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:18.328947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:18.365690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431d1b-e2f0-4d4b-ac1e-9b9b3a6d9d74 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:18.403725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47b641-1c62-400d-a103-d309d67cc5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:18.404672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47b641-1c62-400d-a103-d309d67cc5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.404969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47b641-1c62-400d-a103-d309d67cc5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.405838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47b641-1c62-400d-a103-d309d67cc5f7 None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.407070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47b641-1c62-400d-a103-d309d67cc5f7 None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:18.408540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47b641-1c62-400d-a103-d309d67cc5f7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:18.410480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 53/108] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:18 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:18.415732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47b641-1c62-400d-a103-d309d67cc5f7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:18.523087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47b641-1c62-400d-a103-d309d67cc5f7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:18.559262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbffda-940e-4cd1-84dc-6fe5760a192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:18.560070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbffda-940e-4cd1-84dc-6fe5760a192f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.560413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbffda-940e-4cd1-84dc-6fe5760a192f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.560706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbffda-940e-4cd1-84dc-6fe5760a192f None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.561994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbffda-940e-4cd1-84dc-6fe5760a192f None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:18.563705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbffda-940e-4cd1-84dc-6fe5760a192f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:18.565464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 54/109] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:18 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:18.571151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbffda-940e-4cd1-84dc-6fe5760a192f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:18.583049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:18.631138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bc00-83e1-4a41-acaf-0a4998af0228 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:18.685343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbffda-940e-4cd1-84dc-6fe5760a192f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:18.723775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded23fa1-87b9-49e9-9a7b-4c1248a5e2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:18.726732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded23fa1-87b9-49e9-9a7b-4c1248a5e2ae None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.727839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e83f06-949e-4e3b-bfb3-70d63f3a63be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:18.728757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83f06-949e-4e3b-bfb3-70d63f3a63be None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.728861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded23fa1-87b9-49e9-9a7b-4c1248a5e2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.729173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded23fa1-87b9-49e9-9a7b-4c1248a5e2ae None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b/users/9a28cf8a3ada43b49924f24ae6da72e0/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.731011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83f06-949e-4e3b-bfb3-70d63f3a63be None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.731299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83f06-949e-4e3b-bfb3-70d63f3a63be None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b/users/9a28cf8a3ada43b49924f24ae6da72e0/roles/fc5d6a08fc2048f19c14f691e35cd41c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.732728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded23fa1-87b9-49e9-9a7b-4c1248a5e2ae None admin] RBAC: Authorizing `identity:create_grant(project_id=cd271be4aaeb4efab5ef86157a14122b, user_id=9a28cf8a3ada43b49924f24ae6da72e0, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:18.734868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded23fa1-87b9-49e9-9a7b-4c1248a5e2ae None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:18.739280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e83f06-949e-4e3b-bfb3-70d63f3a63be None admin] RBAC: Authorizing `identity:create_grant(project_id=cd271be4aaeb4efab5ef86157a14122b, user_id=9a28cf8a3ada43b49924f24ae6da72e0, role_id=fc5d6a08fc2048f19c14f691e35cd41c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:18.740828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e83f06-949e-4e3b-bfb3-70d63f3a63be None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:18.814470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 55/110] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:18 2022] PUT /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b/users/9a28cf8a3ada43b49924f24ae6da72e0/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:16:18.815602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b63a-14de-42a7-93c3-7765d03bdadb None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.815923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b63a-14de-42a7-93c3-7765d03bdadb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.816265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b63a-14de-42a7-93c3-7765d03bdadb None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.817610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 56/111] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:18.823672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec580f-cd0a-48e1-8b0a-9346b124af37 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.823964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec580f-cd0a-48e1-8b0a-9346b124af37 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.824251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec580f-cd0a-48e1-8b0a-9346b124af37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.835040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 56/112] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:18 2022] PUT /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b/users/9a28cf8a3ada43b49924f24ae6da72e0/roles/fc5d6a08fc2048f19c14f691e35cd41c => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:18.859674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bcec580f-cd0a-48e1-8b0a-9346b124af37 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:18.860968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bcec580f-cd0a-48e1-8b0a-9346b124af37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:18.882128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43cf5-ccb8-4664-8943-5a0e8d588f8a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.882425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43cf5-ccb8-4664-8943-5a0e8d588f8a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.882702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43cf5-ccb8-4664-8943-5a0e8d588f8a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.884309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 57/113] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:18.912189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7102c-0555-41fe-9351-df0d9b3fc158 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:18.912522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7102c-0555-41fe-9351-df0d9b3fc158 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:18.912765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7102c-0555-41fe-9351-df0d9b3fc158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:18.966862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e5d7102c-0555-41fe-9351-df0d9b3fc158 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:18.968802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e5d7102c-0555-41fe-9351-df0d9b3fc158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:18.998014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:19.009283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 57/114] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:19.011824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19a362-928e-47f4-98e0-c7008a90001a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.012749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19a362-928e-47f4-98e0-c7008a90001a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.013923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19a362-928e-47f4-98e0-c7008a90001a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.016239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 58/115] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:19.018323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ea2fe-a181-428f-8f63-cf2e7f099884 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.019340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ea2fe-a181-428f-8f63-cf2e7f099884 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.020667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ea2fe-a181-428f-8f63-cf2e7f099884 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.030043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 59/116] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:19.033268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0455f4-2264-403f-ac05-3b6393197cbc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.033736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0455f4-2264-403f-ac05-3b6393197cbc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.034126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0455f4-2264-403f-ac05-3b6393197cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.054343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:19.054744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bd0455f4-2264-403f-ac05-3b6393197cbc None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:19.057817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bd0455f4-2264-403f-ac05-3b6393197cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:19.071678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 58/117] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:19.073450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38f19-3f21-488d-904e-7ecfba0e675b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.073897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38f19-3f21-488d-904e-7ecfba0e675b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.074331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b38f19-3f21-488d-904e-7ecfba0e675b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.095318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-88b38f19-3f21-488d-904e-7ecfba0e675b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:19.097221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-88b38f19-3f21-488d-904e-7ecfba0e675b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:19.163191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 60/118] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:19.164722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0455f4-2264-403f-ac05-3b6393197cbc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:19.188084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 59/119] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:19.188958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b38f19-3f21-488d-904e-7ecfba0e675b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:19.281851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0455f4-2264-403f-ac05-3b6393197cbc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:19.312575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:19.324362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7f250-f6f5-4164-8066-2fedad041ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:19.325269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f250-f6f5-4164-8066-2fedad041ccb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.325630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f250-f6f5-4164-8066-2fedad041ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.325977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7f250-f6f5-4164-8066-2fedad041ccb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.327508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b38f19-3f21-488d-904e-7ecfba0e675b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:19.337867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7f250-f6f5-4164-8066-2fedad041ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:19.345666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-39b7f250-f6f5-4164-8066-2fedad041ccb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:19.347508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:19.348532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:19.349489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:19.351130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 61/120] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:19.355990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7f250-f6f5-4164-8066-2fedad041ccb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:19.377195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bf65f-5d4d-42e6-91c0-a6e6550ed730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:19.378638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bf65f-5d4d-42e6-91c0-a6e6550ed730 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.378998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bf65f-5d4d-42e6-91c0-a6e6550ed730 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.379374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bf65f-5d4d-42e6-91c0-a6e6550ed730 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.380957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bf65f-5d4d-42e6-91c0-a6e6550ed730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:19.382471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bf65f-5d4d-42e6-91c0-a6e6550ed730 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:19.389964 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 60/121] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:19.390696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bf65f-5d4d-42e6-91c0-a6e6550ed730 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:19.474207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7f250-f6f5-4164-8066-2fedad041ccb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:19.497226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bf65f-5d4d-42e6-91c0-a6e6550ed730 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:19.513387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f2764-2238-4fef-975c-aacfb030a058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:19.514188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f2764-2238-4fef-975c-aacfb030a058 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.515032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f2764-2238-4fef-975c-aacfb030a058 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.515324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f2764-2238-4fef-975c-aacfb030a058 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.526267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f2764-2238-4fef-975c-aacfb030a058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-986f2764-2238-4fef-975c-aacfb030a058 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:19.534367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:19.536599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:19.538680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:19.540089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ab7a-c183-4bd8-a704-c66f6e0d356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:19.540089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ab7a-c183-4bd8-a704-c66f6e0d356c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.540089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ab7a-c183-4bd8-a704-c66f6e0d356c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.540089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4ab7a-c183-4bd8-a704-c66f6e0d356c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.540701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 62/122] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:19.549162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f2764-2238-4fef-975c-aacfb030a058 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:19.549412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4ab7a-c183-4bd8-a704-c66f6e0d356c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e5a4ab7a-c183-4bd8-a704-c66f6e0d356c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:19.559604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:19.562209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 61/123] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:19.564166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ab7a-c183-4bd8-a704-c66f6e0d356c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:19.594892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:19.652467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f2764-2238-4fef-975c-aacfb030a058 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:19.653629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:19.701053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4ab7a-c183-4bd8-a704-c66f6e0d356c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:19.710262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:19.711293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.711798 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.712351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.713330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:19.713835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:19.715735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:19.725903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 63/124] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:19.727203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:19.747422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5286-533b-49fc-bb33-229da93cf51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:19.751968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5286-533b-49fc-bb33-229da93cf51e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.752260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5286-533b-49fc-bb33-229da93cf51e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.753075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5286-533b-49fc-bb33-229da93cf51e None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.753480 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e5286-533b-49fc-bb33-229da93cf51e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:19.754944 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e5286-533b-49fc-bb33-229da93cf51e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:19.844159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9264-b0cc-4c69-9b60-4a22800e0f6a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:19.857368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:19.865637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 62/125] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:16:19 2022] POST /identity/v3/projects => generated 302 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:19.872109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5286-533b-49fc-bb33-229da93cf51e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:19.907149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:19.908284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:19.908762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:19.909357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:19.910587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:19.911398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:19.913264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:19.938840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 64/126] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:19.940245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:20.053288 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:20.070518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5286-533b-49fc-bb33-229da93cf51e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:20.145985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:20.146577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:20.146910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:20.147189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:20.151922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:20.152246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:20.153734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:20.164752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b44b18-6d0a-4bd7-9235-a0f7e871b048 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:20.165259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 63/127] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:20.165832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:20.229926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13353bad-14c2-45b4-8fe8-7e63017317bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:20.230831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13353bad-14c2-45b4-8fe8-7e63017317bb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:20.231121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13353bad-14c2-45b4-8fe8-7e63017317bb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:20.231405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13353bad-14c2-45b4-8fe8-7e63017317bb None admin] PATH_INFO: `/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:20.233098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13353bad-14c2-45b4-8fe8-7e63017317bb None admin] RBAC: Authorizing `identity:get_user(user_id=f454c88a6bba4ab9ba311bbbd1eb0425)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:20.235327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13353bad-14c2-45b4-8fe8-7e63017317bb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:20.244681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 65/128] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:19 2022] GET /identity/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425 => generated 247 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:20.245875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13353bad-14c2-45b4-8fe8-7e63017317bb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:20.302538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5e99c-fe8c-4dfc-94d9-ef13be2346e6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:20.315312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:20.340451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301401d-04bb-4d63-bcb0-b48942bffcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:20.342046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301401d-04bb-4d63-bcb0-b48942bffcff None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:20.342582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301401d-04bb-4d63-bcb0-b48942bffcff None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:20.343054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301401d-04bb-4d63-bcb0-b48942bffcff None admin] PATH_INFO: `/v3/users/2d6ad4bad81247fbbdb2169047f10eb9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:20.362344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301401d-04bb-4d63-bcb0-b48942bffcff None admin] RBAC: Authorizing `identity:get_user(user_id=2d6ad4bad81247fbbdb2169047f10eb9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:20.364258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301401d-04bb-4d63-bcb0-b48942bffcff None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:20.366233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 64/129] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:20 2022] GET /identity/v3/users/2d6ad4bad81247fbbdb2169047f10eb9 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:20.367188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301401d-04bb-4d63-bcb0-b48942bffcff None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:20.397791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:20.404152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13353bad-14c2-45b4-8fe8-7e63017317bb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:20.445651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db74a-74fc-4b0b-9f11-1cc7b5f89329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:20.446604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db74a-74fc-4b0b-9f11-1cc7b5f89329 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:20.446934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db74a-74fc-4b0b-9f11-1cc7b5f89329 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:20.447222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db74a-74fc-4b0b-9f11-1cc7b5f89329 None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:20.448733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677db74a-74fc-4b0b-9f11-1cc7b5f89329 None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:20.450235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677db74a-74fc-4b0b-9f11-1cc7b5f89329 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:20.452308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 66/130] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:20 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:20.453029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db74a-74fc-4b0b-9f11-1cc7b5f89329 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:20.503831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301401d-04bb-4d63-bcb0-b48942bffcff None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:20.563586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db74a-74fc-4b0b-9f11-1cc7b5f89329 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:20.569818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22991e9d-6d44-4422-a511-97801c0ae668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:20.570938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991e9d-6d44-4422-a511-97801c0ae668 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:20.571420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991e9d-6d44-4422-a511-97801c0ae668 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:20.571918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991e9d-6d44-4422-a511-97801c0ae668 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:20.573136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:20.573945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22991e9d-6d44-4422-a511-97801c0ae668 None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:20.575653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22991e9d-6d44-4422-a511-97801c0ae668 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:20.577609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 65/131] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:20 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:20.579699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22991e9d-6d44-4422-a511-97801c0ae668 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:20.599978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a95b7-cf26-438c-afc7-ae536aaa7a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:20.600638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a95b7-cf26-438c-afc7-ae536aaa7a5e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:20.600967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a95b7-cf26-438c-afc7-ae536aaa7a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:20.601284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a95b7-cf26-438c-afc7-ae536aaa7a5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:20.675796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a95b7-cf26-438c-afc7-ae536aaa7a5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:20.677241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a95b7-cf26-438c-afc7-ae536aaa7a5e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:20.782816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 67/132] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:20.783448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a95b7-cf26-438c-afc7-ae536aaa7a5e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:20.794393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:20.806257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:20.838632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22991e9d-6d44-4422-a511-97801c0ae668 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:20.886158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a95b7-cf26-438c-afc7-ae536aaa7a5e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:20.908432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ab72-22c9-4083-9427-dd7523571096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:20.909338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ab72-22c9-4083-9427-dd7523571096 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:20.909628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ab72-22c9-4083-9427-dd7523571096 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:20.909910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7ab72-22c9-4083-9427-dd7523571096 None admin] PATH_INFO: `/v3/projects/e75edf2a51ec4bf68976b61eb3e42ab6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:20.911644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ab72-22c9-4083-9427-dd7523571096 None admin] RBAC: Authorizing `identity:get_project(project_id=e75edf2a51ec4bf68976b61eb3e42ab6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:20.914721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7ab72-22c9-4083-9427-dd7523571096 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:20.914721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 66/133] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:20 2022] GET /identity/v3/projects/e75edf2a51ec4bf68976b61eb3e42ab6 => generated 312 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:20.915534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ab72-22c9-4083-9427-dd7523571096 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:20.962471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec269b84-35a6-4e57-a2b3-3a8651946bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:20.963289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec269b84-35a6-4e57-a2b3-3a8651946bb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:20.963571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec269b84-35a6-4e57-a2b3-3a8651946bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:20.963847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec269b84-35a6-4e57-a2b3-3a8651946bb0 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b/users/f454c88a6bba4ab9ba311bbbd1eb0425/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:20.970690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec269b84-35a6-4e57-a2b3-3a8651946bb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd271be4aaeb4efab5ef86157a14122b, user_id=f454c88a6bba4ab9ba311bbbd1eb0425, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:20.972487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec269b84-35a6-4e57-a2b3-3a8651946bb0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:21.025426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7ab72-22c9-4083-9427-dd7523571096 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:21.064827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 68/134] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:20 2022] PUT /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b/users/f454c88a6bba4ab9ba311bbbd1eb0425/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:21.070302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec269b84-35a6-4e57-a2b3-3a8651946bb0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:21.079329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138599e-46c7-4c80-a633-c2bfb02da9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:21.079329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138599e-46c7-4c80-a633-c2bfb02da9fb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:21.079329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138599e-46c7-4c80-a633-c2bfb02da9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:21.079663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138599e-46c7-4c80-a633-c2bfb02da9fb None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:21.080789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138599e-46c7-4c80-a633-c2bfb02da9fb None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:21.082299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138599e-46c7-4c80-a633-c2bfb02da9fb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:21.085131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 67/135] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:20 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:21.106173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138599e-46c7-4c80-a633-c2bfb02da9fb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:21.176458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:21.216317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:21.301795 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec269b84-35a6-4e57-a2b3-3a8651946bb0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:21.392612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9681df-6a4c-4661-9953-f4c9a5de9c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:21.393539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9681df-6a4c-4661-9953-f4c9a5de9c7a None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:21.393822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9681df-6a4c-4661-9953-f4c9a5de9c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:21.394099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9681df-6a4c-4661-9953-f4c9a5de9c7a None admin] PATH_INFO: `/v3/projects/e75edf2a51ec4bf68976b61eb3e42ab6/users/9a28cf8a3ada43b49924f24ae6da72e0/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:21.396621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138599e-46c7-4c80-a633-c2bfb02da9fb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:21.400022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9681df-6a4c-4661-9953-f4c9a5de9c7a None admin] RBAC: Authorizing `identity:create_grant(project_id=e75edf2a51ec4bf68976b61eb3e42ab6, user_id=9a28cf8a3ada43b49924f24ae6da72e0, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:21.401518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9681df-6a4c-4661-9953-f4c9a5de9c7a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:21.441250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a937e6f-af80-4ef5-aad7-2d44f86ed73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:21.441925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a937e6f-af80-4ef5-aad7-2d44f86ed73f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:21.442218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a937e6f-af80-4ef5-aad7-2d44f86ed73f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:21.442501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a937e6f-af80-4ef5-aad7-2d44f86ed73f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:21.443338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a937e6f-af80-4ef5-aad7-2d44f86ed73f None admin] RBAC: Adding query filter params (role.id=86bf554ccc4b46ee91870389659035e3, scope.project.id=cd271be4aaeb4efab5ef86157a14122b, user.id=2d6ad4bad81247fbbdb2169047f10eb9) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:21.443659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a937e6f-af80-4ef5-aad7-2d44f86ed73f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:21.445132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a937e6f-af80-4ef5-aad7-2d44f86ed73f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:21.451180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 68/136] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:21 2022] GET /identity/v3/role_assignments?user.id=2d6ad4bad81247fbbdb2169047f10eb9&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=86bf554ccc4b46ee91870389659035e3 => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:21.452852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd309f-ad84-4051-9cdf-bfb892adc231 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:21.453248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd309f-ad84-4051-9cdf-bfb892adc231 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:21.453547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd309f-ad84-4051-9cdf-bfb892adc231 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:21.454843 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 69/137] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:21.497041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 69/138] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:21 2022] PUT /identity/v3/projects/e75edf2a51ec4bf68976b61eb3e42ab6/users/9a28cf8a3ada43b49924f24ae6da72e0/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:21.497377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d69d69-ba0f-49ee-8cd3-3a0c4b15d8b5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:21.497377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d69d69-ba0f-49ee-8cd3-3a0c4b15d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:21.497377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d69d69-ba0f-49ee-8cd3-3a0c4b15d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:21.497377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-12d69d69-ba0f-49ee-8cd3-3a0c4b15d8b5 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:21.497377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-12d69d69-ba0f-49ee-8cd3-3a0c4b15d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:21.573811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 70/139] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:21.610058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9681df-6a4c-4661-9953-f4c9a5de9c7a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:21.695181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a847a-8567-4da7-9f79-1e66fd5a6839 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:21.695522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a847a-8567-4da7-9f79-1e66fd5a6839 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:21.695768 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a847a-8567-4da7-9f79-1e66fd5a6839 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:21.697538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 71/140] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:21.705411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da28f-a62d-4e77-bee2-130d5a56b59f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:21.705695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da28f-a62d-4e77-bee2-130d5a56b59f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:21.705982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91da28f-a62d-4e77-bee2-130d5a56b59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:21.736206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e91da28f-a62d-4e77-bee2-130d5a56b59f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:21.737649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e91da28f-a62d-4e77-bee2-130d5a56b59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:21.760199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:21.799006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9681df-6a4c-4661-9953-f4c9a5de9c7a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:21.833316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:21.841732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2e88c-89d9-4705-ac70-4404549eb5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:21.847202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2e88c-89d9-4705-ac70-4404549eb5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:21.847202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2e88c-89d9-4705-ac70-4404549eb5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:21.847202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2e88c-89d9-4705-ac70-4404549eb5f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:21.853780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 72/141] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 28 03:16:21.856066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2e88c-89d9-4705-ac70-4404549eb5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c8e2e88c-89d9-4705-ac70-4404549eb5f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:21.864020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:21.864931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:21.866138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 70/142] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:21.874451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91da28f-a62d-4e77-bee2-130d5a56b59f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:21.881954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2e88c-89d9-4705-ac70-4404549eb5f2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:21.985621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91da28f-a62d-4e77-bee2-130d5a56b59f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:22.023401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:22.024058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:22.024353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:22.024642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:22.025434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:22.025752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:22.027157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:22.041792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 73/143] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:22.048553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:22.073523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:22.107451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2e88c-89d9-4705-ac70-4404549eb5f2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:22.193874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ad20d-2f7f-4f68-9277-a413e22e1f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:22.195216 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad20d-2f7f-4f68-9277-a413e22e1f96 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:22.196831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad20d-2f7f-4f68-9277-a413e22e1f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:22.197914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad20d-2f7f-4f68-9277-a413e22e1f96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:22.208916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03d69b-bce6-47fd-a071-1d0dbd30eb73 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:22.221266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ad20d-2f7f-4f68-9277-a413e22e1f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-747ad20d-2f7f-4f68-9277-a413e22e1f96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:22.232654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:22.234135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:22.234877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:22.240777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 71/144] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:16:22.246851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ad20d-2f7f-4f68-9277-a413e22e1f96 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:22.294552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5baac-877b-4823-b017-b1fd44125834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:22.296337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5baac-877b-4823-b017-b1fd44125834 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:22.296836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5baac-877b-4823-b017-b1fd44125834 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:22.297337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5baac-877b-4823-b017-b1fd44125834 None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:22.299743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5baac-877b-4823-b017-b1fd44125834 None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:22.301442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5baac-877b-4823-b017-b1fd44125834 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:22.303283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 74/145] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:22 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:22.311590 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5baac-877b-4823-b017-b1fd44125834 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:22.353440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:22.447842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ad20d-2f7f-4f68-9277-a413e22e1f96 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:22.475125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:22.502287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:22.502964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:22.503246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:22.503526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:22.504282 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:22.504583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:22.506000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:22.518663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5baac-877b-4823-b017-b1fd44125834 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:22.523242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 72/146] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:22.534092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:22.584222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cad65-7c19-45a1-bb93-383008a70b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:22.584790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cad65-7c19-45a1-bb93-383008a70b9b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:22.585073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cad65-7c19-45a1-bb93-383008a70b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:22.585372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4cad65-7c19-45a1-bb93-383008a70b9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:22.706302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a2726-2d7f-47c5-a250-2b6484ee4ed7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:22.708386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cad65-7c19-45a1-bb93-383008a70b9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:22.709896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4cad65-7c19-45a1-bb93-383008a70b9b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:22.727344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:22.758316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ef466-fd92-4381-9dc5-c3480061890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:22.759198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef466-fd92-4381-9dc5-c3480061890b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:22.759477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef466-fd92-4381-9dc5-c3480061890b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:22.759752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef466-fd92-4381-9dc5-c3480061890b None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:22.761132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ef466-fd92-4381-9dc5-c3480061890b None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:22.762643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ef466-fd92-4381-9dc5-c3480061890b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:22.764783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 73/147] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:22 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:22.770078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ef466-fd92-4381-9dc5-c3480061890b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:22.866092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:22.920840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 75/148] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:22.926820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cad65-7c19-45a1-bb93-383008a70b9b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:22.984162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ef466-fd92-4381-9dc5-c3480061890b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:23.045927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fffb92-e6a4-4d7a-b2b9-1a98042493d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:23.046699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fffb92-e6a4-4d7a-b2b9-1a98042493d6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:23.046984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fffb92-e6a4-4d7a-b2b9-1a98042493d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:23.047261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fffb92-e6a4-4d7a-b2b9-1a98042493d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:23.114783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4cad65-7c19-45a1-bb93-383008a70b9b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:23.127722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:23.164348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fffb92-e6a4-4d7a-b2b9-1a98042493d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:23.165862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fffb92-e6a4-4d7a-b2b9-1a98042493d6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:23.174202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bddfad5-14e8-42d1-85f0-3c3bb64ac63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:23.175067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddfad5-14e8-42d1-85f0-3c3bb64ac63b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:23.175346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddfad5-14e8-42d1-85f0-3c3bb64ac63b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:23.175620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bddfad5-14e8-42d1-85f0-3c3bb64ac63b None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:23.177171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddfad5-14e8-42d1-85f0-3c3bb64ac63b None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:23.178625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bddfad5-14e8-42d1-85f0-3c3bb64ac63b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:23.185239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 76/149] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:22 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:23.187042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a3ffa-1677-4c78-a72d-1d9dd7213a12 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:23.187327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a3ffa-1677-4c78-a72d-1d9dd7213a12 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:23.187602 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a3ffa-1677-4c78-a72d-1d9dd7213a12 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:23.189023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 77/150] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:23.192565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a3ffa-1677-4c78-a72d-1d9dd7213a12 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:23.252529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:23.305679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 74/151] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 03:16:23.307215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc56b8-c326-4372-8f93-8b256263171c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:23.307515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc56b8-c326-4372-8f93-8b256263171c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:23.307797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc56b8-c326-4372-8f93-8b256263171c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:23.334990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a3ffa-1677-4c78-a72d-1d9dd7213a12 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:23.346252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-23cc56b8-c326-4372-8f93-8b256263171c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:23.347750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-23cc56b8-c326-4372-8f93-8b256263171c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:23.396657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:23.397255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:23.397537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:23.398511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:23.399560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] RBAC: Adding query filter params (role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=cd271be4aaeb4efab5ef86157a14122b, user.id=9a28cf8a3ada43b49924f24ae6da72e0) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:23.399872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:23.402486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:23.420271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 78/152] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:23 2022] GET /identity/v3/role_assignments?user.id=9a28cf8a3ada43b49924f24ae6da72e0&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:23.421163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:23.451310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 75/153] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:23.503697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc56b8-c326-4372-8f93-8b256263171c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:23.515428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:23.598445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6e479-66b5-4c5a-ad8d-96f202591440 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:23.609537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:23.658105 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a600-9bbe-49eb-a4c9-d95a4004f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:23.658901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a600-9bbe-49eb-a4c9-d95a4004f8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:23.659180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a600-9bbe-49eb-a4c9-d95a4004f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:23.659512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2a600-9bbe-49eb-a4c9-d95a4004f8c9 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:23.661531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2a600-9bbe-49eb-a4c9-d95a4004f8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:23.663292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2a600-9bbe-49eb-a4c9-d95a4004f8c9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:23.665020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 79/154] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:23 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:23.675224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a600-9bbe-49eb-a4c9-d95a4004f8c9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:23.723301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc56b8-c326-4372-8f93-8b256263171c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:23.785676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cf885-5ae9-4e19-b06b-143b8c38b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:23.786750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cf885-5ae9-4e19-b06b-143b8c38b23c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:23.787870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cf885-5ae9-4e19-b06b-143b8c38b23c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:23.788156 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cf885-5ae9-4e19-b06b-143b8c38b23c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:23.789524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cf885-5ae9-4e19-b06b-143b8c38b23c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:23.792383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cf885-5ae9-4e19-b06b-143b8c38b23c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:23.794067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 76/155] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:23.799800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cf885-5ae9-4e19-b06b-143b8c38b23c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:23.915803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:23.917216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2a600-9bbe-49eb-a4c9-d95a4004f8c9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:23.949762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cf885-5ae9-4e19-b06b-143b8c38b23c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:23.993498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3658c5-ba56-4096-b17a-8a904e4efbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:23.994483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3658c5-ba56-4096-b17a-8a904e4efbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:23.994925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3658c5-ba56-4096-b17a-8a904e4efbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:23.995348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3658c5-ba56-4096-b17a-8a904e4efbb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:23.997377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3658c5-ba56-4096-b17a-8a904e4efbb9 None admin] RBAC: Adding query filter params (role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=cd271be4aaeb4efab5ef86157a14122b, user.id=9a28cf8a3ada43b49924f24ae6da72e0) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:23.998584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3658c5-ba56-4096-b17a-8a904e4efbb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:24.000973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3658c5-ba56-4096-b17a-8a904e4efbb9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:24.013418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f5c4f-94cd-493f-9794-6dbd8ab399ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:24.015830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 80/156] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:23 2022] GET /identity/v3/role_assignments?user.id=9a28cf8a3ada43b49924f24ae6da72e0&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:24.016714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f5c4f-94cd-493f-9794-6dbd8ab399ec None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.017246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f5c4f-94cd-493f-9794-6dbd8ab399ec None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.017689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f5c4f-94cd-493f-9794-6dbd8ab399ec None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.018575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f5c4f-94cd-493f-9794-6dbd8ab399ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:24.020286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f5c4f-94cd-493f-9794-6dbd8ab399ec None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:24.124707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 77/157] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:16:23 2022] POST /identity/v3/users => generated 283 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:24.427970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228154b4-b510-4f4c-9590-aa4b66c754ac None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.428559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228154b4-b510-4f4c-9590-aa4b66c754ac None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.428992 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228154b4-b510-4f4c-9590-aa4b66c754ac None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.430597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 81/158] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:24.436015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516715b-27e2-46cf-9be7-0245a3458c6e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.437072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516715b-27e2-46cf-9be7-0245a3458c6e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.437642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516715b-27e2-46cf-9be7-0245a3458c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.456089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3516715b-27e2-46cf-9be7-0245a3458c6e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:24.457642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3516715b-27e2-46cf-9be7-0245a3458c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:24.519920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca55d-7fdf-4a50-bac8-ba3777d87318 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.520565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca55d-7fdf-4a50-bac8-ba3777d87318 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.520849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ca55d-7fdf-4a50-bac8-ba3777d87318 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.522426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 82/159] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:24.527751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e6415-3668-47d9-b878-eef4a088e49f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.528039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e6415-3668-47d9-b878-eef4a088e49f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.528318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e6415-3668-47d9-b878-eef4a088e49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.555838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1f0e6415-3668-47d9-b878-eef4a088e49f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:24.557038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1f0e6415-3668-47d9-b878-eef4a088e49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:24.558495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 78/160] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:24.586102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516715b-27e2-46cf-9be7-0245a3458c6e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:24.597275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:24.635525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 83/161] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:24.661031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:24.661961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e6415-3668-47d9-b878-eef4a088e49f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:24.713727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516715b-27e2-46cf-9be7-0245a3458c6e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:24.755072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c99f0-86a9-455d-ab69-67bd8488665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:24.755914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c99f0-86a9-455d-ab69-67bd8488665a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.756201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c99f0-86a9-455d-ab69-67bd8488665a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.756481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c99f0-86a9-455d-ab69-67bd8488665a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.769357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c99f0-86a9-455d-ab69-67bd8488665a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e77c99f0-86a9-455d-ab69-67bd8488665a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:24.777544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:24.778504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:24.779269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 79/162] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:16:24.780968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da78fe-95f7-4e3c-9264-d2939edf2d49 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.780968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da78fe-95f7-4e3c-9264-d2939edf2d49 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.781175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da78fe-95f7-4e3c-9264-d2939edf2d49 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.782375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 80/163] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:24.787369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da78fe-95f7-4e3c-9264-d2939edf2d49 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:24.799506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e6415-3668-47d9-b878-eef4a088e49f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:24.853666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b01be6b-8ba5-4bd5-a706-449f371ce85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:24.854808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01be6b-8ba5-4bd5-a706-449f371ce85e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.855197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01be6b-8ba5-4bd5-a706-449f371ce85e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.855872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01be6b-8ba5-4bd5-a706-449f371ce85e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.870196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b01be6b-8ba5-4bd5-a706-449f371ce85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4b01be6b-8ba5-4bd5-a706-449f371ce85e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:24.880330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:24.881430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 84/164] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:16:24.882876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73971da7-ae08-49b7-b654-297fec65627d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.883797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73971da7-ae08-49b7-b654-297fec65627d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.883797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73971da7-ae08-49b7-b654-297fec65627d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.905990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-73971da7-ae08-49b7-b654-297fec65627d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:24.907133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-73971da7-ae08-49b7-b654-297fec65627d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:24.936679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da78fe-95f7-4e3c-9264-d2939edf2d49 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:24.939348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:24.965363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:24.981234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:24.991548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:24.991548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:24.991548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:24.991548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:24.991548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:24.991548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:24.994579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 85/165] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:24.995530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73971da7-ae08-49b7-b654-297fec65627d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:25.004459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 81/166] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:25.011207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:25.190077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd35ea-23e7-4d67-a277-a48914e1c605 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:25.195673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73971da7-ae08-49b7-b654-297fec65627d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:25.260338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f575922-580f-4e2e-ac52-5384c8a41af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:25.261252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f575922-580f-4e2e-ac52-5384c8a41af6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:25.261535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f575922-580f-4e2e-ac52-5384c8a41af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:25.261810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f575922-580f-4e2e-ac52-5384c8a41af6 None admin] PATH_INFO: `/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:25.263511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f575922-580f-4e2e-ac52-5384c8a41af6 None admin] RBAC: Authorizing `identity:get_user(user_id=f454c88a6bba4ab9ba311bbbd1eb0425)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:25.265566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f575922-580f-4e2e-ac52-5384c8a41af6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:25.266057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:25.266986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:25.267206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 82/167] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:25 2022] GET /identity/v3/users/f454c88a6bba4ab9ba311bbbd1eb0425 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:25.267670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:25.268158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:25.269152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:25.269664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:25.269870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f575922-580f-4e2e-ac52-5384c8a41af6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:25.271575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:25.281017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 86/168] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:25.282169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:25.308446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:25.362780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:25.411801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f575922-580f-4e2e-ac52-5384c8a41af6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:25.447970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822a63f-9955-482a-b237-67341cbb7328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:25.449539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822a63f-9955-482a-b237-67341cbb7328 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:25.450080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822a63f-9955-482a-b237-67341cbb7328 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:25.450554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822a63f-9955-482a-b237-67341cbb7328 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:25.462125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822a63f-9955-482a-b237-67341cbb7328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:25.463225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4fd73-ffff-4d7b-ba3d-17c903a7faf7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3822a63f-9955-482a-b237-67341cbb7328 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:25.470545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:25.471697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:25.472468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:25.474866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 83/169] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:25.475860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822a63f-9955-482a-b237-67341cbb7328 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:25.558452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad861e8a-367c-4900-b690-c05483185eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:25.559032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861e8a-367c-4900-b690-c05483185eed None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:25.559322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861e8a-367c-4900-b690-c05483185eed None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:25.559603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad861e8a-367c-4900-b690-c05483185eed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:25.584898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822a63f-9955-482a-b237-67341cbb7328 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:25.601460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:25.621036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf366c3-b138-4327-8462-43bf7b637c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:25.622299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf366c3-b138-4327-8462-43bf7b637c75 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:25.622789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf366c3-b138-4327-8462-43bf7b637c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:25.623278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf366c3-b138-4327-8462-43bf7b637c75 None admin] PATH_INFO: `/v3/users/9a28cf8a3ada43b49924f24ae6da72e0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:25.624734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf366c3-b138-4327-8462-43bf7b637c75 None admin] RBAC: Authorizing `identity:get_user(user_id=9a28cf8a3ada43b49924f24ae6da72e0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:25.626526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf366c3-b138-4327-8462-43bf7b637c75 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:25.628504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 84/170] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:25 2022] GET /identity/v3/users/9a28cf8a3ada43b49924f24ae6da72e0 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:25.629522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf366c3-b138-4327-8462-43bf7b637c75 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:25.663546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad861e8a-367c-4900-b690-c05483185eed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:25.664979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad861e8a-367c-4900-b690-c05483185eed None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:25.688380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:25.747034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf366c3-b138-4327-8462-43bf7b637c75 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:25.784645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:25.785541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:25.785950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:25.786331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:25.787247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:25.787655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:25.789164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:25.792434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 87/171] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:25.793335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad861e8a-367c-4900-b690-c05483185eed None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:25.798413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 85/172] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:25.799218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:25.854103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:25.954497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad861e8a-367c-4900-b690-c05483185eed None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:26.013461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:26.013752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40015f00-4f91-4753-a45f-50e35860ce0a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:26.040186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4f28b-e979-463a-9674-0d07e879e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:26.041167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f28b-e979-463a-9674-0d07e879e718 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:26.041604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f28b-e979-463a-9674-0d07e879e718 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:26.042019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f28b-e979-463a-9674-0d07e879e718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:26.087777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb84b1-6f23-4879-8a3a-f41079226be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:26.088647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb84b1-6f23-4879-8a3a-f41079226be9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:26.088931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb84b1-6f23-4879-8a3a-f41079226be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:26.089234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb84b1-6f23-4879-8a3a-f41079226be9 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:26.090554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb84b1-6f23-4879-8a3a-f41079226be9 None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:26.094176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eb84b1-6f23-4879-8a3a-f41079226be9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:26.095659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 86/173] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:25 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:26.096257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb84b1-6f23-4879-8a3a-f41079226be9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:26.178719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4f28b-e979-463a-9674-0d07e879e718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:26.180120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4f28b-e979-463a-9674-0d07e879e718 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:26.279285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb84b1-6f23-4879-8a3a-f41079226be9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:26.294486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:26.351928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53432972-f62e-472d-a02b-db36c7bac952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:26.352888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53432972-f62e-472d-a02b-db36c7bac952 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:26.353859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53432972-f62e-472d-a02b-db36c7bac952 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:26.354152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53432972-f62e-472d-a02b-db36c7bac952 None admin] PATH_INFO: `/v3/users/2d6ad4bad81247fbbdb2169047f10eb9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:26.355580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53432972-f62e-472d-a02b-db36c7bac952 None admin] RBAC: Authorizing `identity:get_user(user_id=2d6ad4bad81247fbbdb2169047f10eb9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:26.357085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53432972-f62e-472d-a02b-db36c7bac952 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:26.359073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 87/174] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:26 2022] GET /identity/v3/users/2d6ad4bad81247fbbdb2169047f10eb9 => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:26.360259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53432972-f62e-472d-a02b-db36c7bac952 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:26.363520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 88/175] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:26.365005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4f28b-e979-463a-9674-0d07e879e718 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:26.417469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:26.543404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4f28b-e979-463a-9674-0d07e879e718 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:26.546531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53432972-f62e-472d-a02b-db36c7bac952 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:26.585996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e6b3d-1830-49d3-a458-f38cd510e0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:26.586874 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e6b3d-1830-49d3-a458-f38cd510e0ad None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:26.587161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e6b3d-1830-49d3-a458-f38cd510e0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:26.587443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e6b3d-1830-49d3-a458-f38cd510e0ad None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:26.589394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e6b3d-1830-49d3-a458-f38cd510e0ad None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:26.590913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e6b3d-1830-49d3-a458-f38cd510e0ad None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:26.593430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 89/176] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:26 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:26.594007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e6b3d-1830-49d3-a458-f38cd510e0ad None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:26.614872 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:26.615534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:26.615822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:26.616102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:26.616945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=cd271be4aaeb4efab5ef86157a14122b, user.id=f454c88a6bba4ab9ba311bbbd1eb0425) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:26.617292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:26.618778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:26.633149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 88/177] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:26 2022] GET /identity/v3/role_assignments?user.id=f454c88a6bba4ab9ba311bbbd1eb0425&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=bb0756f8af694633b9743a958eb46e7a => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:26.633976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:26.717891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:26.741478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:26.759978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e6b3d-1830-49d3-a458-f38cd510e0ad None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:26.821215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf226f73-6bb6-4d78-8d74-11abea7d62df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:26.822400 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf226f73-6bb6-4d78-8d74-11abea7d62df None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:26.822887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf226f73-6bb6-4d78-8d74-11abea7d62df None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:26.823343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf226f73-6bb6-4d78-8d74-11abea7d62df None admin] PATH_INFO: `/v3/projects/e75edf2a51ec4bf68976b61eb3e42ab6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:26.828037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf226f73-6bb6-4d78-8d74-11abea7d62df None admin] RBAC: Authorizing `identity:get_project(project_id=e75edf2a51ec4bf68976b61eb3e42ab6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:26.831844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf226f73-6bb6-4d78-8d74-11abea7d62df None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:26.834016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 90/178] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:26 2022] GET /identity/v3/projects/e75edf2a51ec4bf68976b61eb3e42ab6 => generated 312 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:26.840182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf226f73-6bb6-4d78-8d74-11abea7d62df None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:26.853152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da3b7b3-070e-426a-acd6-5f01067ce00c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:26.920878 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df01e11-2c51-4e02-b1b5-d66ddecaca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:26.926223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df01e11-2c51-4e02-b1b5-d66ddecaca8c None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:26.927254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df01e11-2c51-4e02-b1b5-d66ddecaca8c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:26.927613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df01e11-2c51-4e02-b1b5-d66ddecaca8c None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b/users/2d6ad4bad81247fbbdb2169047f10eb9/roles/86bf554ccc4b46ee91870389659035e3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:26.930155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df01e11-2c51-4e02-b1b5-d66ddecaca8c None admin] RBAC: Authorizing `identity:create_grant(project_id=cd271be4aaeb4efab5ef86157a14122b, user_id=2d6ad4bad81247fbbdb2169047f10eb9, role_id=86bf554ccc4b46ee91870389659035e3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:26.931717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df01e11-2c51-4e02-b1b5-d66ddecaca8c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:27.048245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf226f73-6bb6-4d78-8d74-11abea7d62df None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:27.055979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 89/179] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:26 2022] PUT /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b/users/2d6ad4bad81247fbbdb2169047f10eb9/roles/86bf554ccc4b46ee91870389659035e3 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:27.112912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91135bf0-23b6-495b-8755-bc226011c1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:27.113954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135bf0-23b6-495b-8755-bc226011c1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.114444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135bf0-23b6-495b-8755-bc226011c1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.114914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135bf0-23b6-495b-8755-bc226011c1d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.116024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91135bf0-23b6-495b-8755-bc226011c1d3 None admin] RBAC: Adding query filter params (role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=e75edf2a51ec4bf68976b61eb3e42ab6, user.id=9a28cf8a3ada43b49924f24ae6da72e0) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:27.116575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91135bf0-23b6-495b-8755-bc226011c1d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:27.118343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91135bf0-23b6-495b-8755-bc226011c1d3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:27.129946 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 91/180] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:26 2022] GET /identity/v3/role_assignments?user.id=9a28cf8a3ada43b49924f24ae6da72e0&scope.project.id=e75edf2a51ec4bf68976b61eb3e42ab6&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:27.134192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa2802-ffbb-4bc2-854e-faaca662e5f2 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.134509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa2802-ffbb-4bc2-854e-faaca662e5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.134773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa2802-ffbb-4bc2-854e-faaca662e5f2 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.136238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 90/181] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:27.142883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f25df-9231-46d1-8f99-a8608fd6322e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.143495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f25df-9231-46d1-8f99-a8608fd6322e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.144009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f25df-9231-46d1-8f99-a8608fd6322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.163782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b10f25df-9231-46d1-8f99-a8608fd6322e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:27.165410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b10f25df-9231-46d1-8f99-a8608fd6322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:27.186301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:27.248880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 92/182] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:27.261750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff016-bde6-4506-8145-e97716337d41 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.262117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff016-bde6-4506-8145-e97716337d41 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.262327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dff016-bde6-4506-8145-e97716337d41 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.264029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 91/183] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:27.270198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2452-cccf-46c6-b36b-271d507e4118 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.270808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2452-cccf-46c6-b36b-271d507e4118 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.271426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2452-cccf-46c6-b36b-271d507e4118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.276505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dff016-bde6-4506-8145-e97716337d41 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:27.292989 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-544e2452-cccf-46c6-b36b-271d507e4118 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:27.294728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-544e2452-cccf-46c6-b36b-271d507e4118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:27.385982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 93/184] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:27.420730 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2452-cccf-46c6-b36b-271d507e4118 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:27.475863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dff016-bde6-4506-8145-e97716337d41 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:27.477631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:27.512530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea220c0-e6d7-4f0e-81c7-e5948a54413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:27.513375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea220c0-e6d7-4f0e-81c7-e5948a54413a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.513663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea220c0-e6d7-4f0e-81c7-e5948a54413a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.513943 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea220c0-e6d7-4f0e-81c7-e5948a54413a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.526418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea220c0-e6d7-4f0e-81c7-e5948a54413a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:27.531866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-bea220c0-e6d7-4f0e-81c7-e5948a54413a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:27.534424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:27.537750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:27.538506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 92/185] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:27.541690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea220c0-e6d7-4f0e-81c7-e5948a54413a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:27.552471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2452-cccf-46c6-b36b-271d507e4118 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:27.604504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7e620-f067-4176-b86c-114cfed4605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:27.606314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e620-f067-4176-b86c-114cfed4605c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.606860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e620-f067-4176-b86c-114cfed4605c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.607566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7e620-f067-4176-b86c-114cfed4605c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.621872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7e620-f067-4176-b86c-114cfed4605c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-82c7e620-f067-4176-b86c-114cfed4605c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:27.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:27.634170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:27.635142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:27.636381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 94/186] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:16:27.637597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7e335-bcb5-42a4-b804-1b7c4bff0ea5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.637889 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7e335-bcb5-42a4-b804-1b7c4bff0ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.638229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7e335-bcb5-42a4-b804-1b7c4bff0ea5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.640095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 95/187] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:27.644533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7e335-bcb5-42a4-b804-1b7c4bff0ea5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:27.668596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea220c0-e6d7-4f0e-81c7-e5948a54413a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:27.705838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209914a6-d3a2-470c-9933-0f4107035aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:27.706504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209914a6-d3a2-470c-9933-0f4107035aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.706797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209914a6-d3a2-470c-9933-0f4107035aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.707078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209914a6-d3a2-470c-9933-0f4107035aa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.707826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209914a6-d3a2-470c-9933-0f4107035aa1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:27.708138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209914a6-d3a2-470c-9933-0f4107035aa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:27.709569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209914a6-d3a2-470c-9933-0f4107035aa1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:27.719201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 93/188] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:27.724900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630afec5-033e-4af5-bb1b-6fba24bf9786 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.725227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630afec5-033e-4af5-bb1b-6fba24bf9786 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.725518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630afec5-033e-4af5-bb1b-6fba24bf9786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.742742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-630afec5-033e-4af5-bb1b-6fba24bf9786 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:27.743926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-630afec5-033e-4af5-bb1b-6fba24bf9786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:27.752393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7e335-bcb5-42a4-b804-1b7c4bff0ea5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:27.763587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:27.787828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:27.788498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.788781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.795054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.795054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:27.795054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:27.795054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:27.804643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 96/189] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:27.804643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:27.807888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:27.818215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 94/190] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:27.822901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630afec5-033e-4af5-bb1b-6fba24bf9786 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:27.914051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798ec34-e3ad-4cba-8c0e-5995ff609bd7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:27.936671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630afec5-033e-4af5-bb1b-6fba24bf9786 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:27.953459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff4c65-515d-42db-8dde-8d76622f226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:27.954331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff4c65-515d-42db-8dde-8d76622f226e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.954616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff4c65-515d-42db-8dde-8d76622f226e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.954897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff4c65-515d-42db-8dde-8d76622f226e None admin] PATH_INFO: `/v3/users/2fa587ae4e84494da315773eef05e33c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.968629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff4c65-515d-42db-8dde-8d76622f226e None admin] RBAC: Authorizing `identity:get_user(user_id=2fa587ae4e84494da315773eef05e33c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:27.970726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff4c65-515d-42db-8dde-8d76622f226e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:27.977060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 97/191] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:27 2022] GET /identity/v3/users/2fa587ae4e84494da315773eef05e33c => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:27.977060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff4c65-515d-42db-8dde-8d76622f226e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:27.978045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ff59d-faa9-4f71-b616-e49960772b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:27.978936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ff59d-faa9-4f71-b616-e49960772b3e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:27.979224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ff59d-faa9-4f71-b616-e49960772b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:27.979502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ff59d-faa9-4f71-b616-e49960772b3e None admin] PATH_INFO: `/v3/users/2fa587ae4e84494da315773eef05e33c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:27.980733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ff59d-faa9-4f71-b616-e49960772b3e None admin] RBAC: Authorizing `identity:get_user(user_id=2fa587ae4e84494da315773eef05e33c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:27.982234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ff59d-faa9-4f71-b616-e49960772b3e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:27.984577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 95/192] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:27 2022] GET /identity/v3/users/2fa587ae4e84494da315773eef05e33c => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:27.986060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ff59d-faa9-4f71-b616-e49960772b3e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:28.019592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:28.050075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:28.092220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff4c65-515d-42db-8dde-8d76622f226e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:28.098047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ff59d-faa9-4f71-b616-e49960772b3e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:28.132798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29fbc8-1fba-43b6-b6c4-ba1ac16ac4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:28.133627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29fbc8-1fba-43b6-b6c4-ba1ac16ac4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:28.133921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29fbc8-1fba-43b6-b6c4-ba1ac16ac4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:28.134200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29fbc8-1fba-43b6-b6c4-ba1ac16ac4f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:28.135367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29fbc8-1fba-43b6-b6c4-ba1ac16ac4f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:28.136809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c29fbc8-1fba-43b6-b6c4-ba1ac16ac4f8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:28.137803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9ee3-2b7e-4a49-beb8-30e2e8e7b3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:28.143187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 98/193] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:28.143187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29fbc8-1fba-43b6-b6c4-ba1ac16ac4f8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:28.143400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9ee3-2b7e-4a49-beb8-30e2e8e7b3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:28.143400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9ee3-2b7e-4a49-beb8-30e2e8e7b3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:28.143400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c9ee3-2b7e-4a49-beb8-30e2e8e7b3a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:28.213838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c9ee3-2b7e-4a49-beb8-30e2e8e7b3a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:28.215568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c9ee3-2b7e-4a49-beb8-30e2e8e7b3a8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:28.239349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:28.245073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29fbc8-1fba-43b6-b6c4-ba1ac16ac4f8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:28.275965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:28.282615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace20d26-bbe0-4353-84be-c1a9e3f03b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:28.283372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20d26-bbe0-4353-84be-c1a9e3f03b24 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:28.283661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20d26-bbe0-4353-84be-c1a9e3f03b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:28.283934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace20d26-bbe0-4353-84be-c1a9e3f03b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:28.318576 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 96/194] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:28.319391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9ee3-2b7e-4a49-beb8-30e2e8e7b3a8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:28.360400 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace20d26-bbe0-4353-84be-c1a9e3f03b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:28.362013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace20d26-bbe0-4353-84be-c1a9e3f03b24 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:28.425647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c9ee3-2b7e-4a49-beb8-30e2e8e7b3a8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:28.453532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:28.465011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11f0a6-cb1e-4f63-ad63-962750596484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:28.465651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11f0a6-cb1e-4f63-ad63-962750596484 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:28.465946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11f0a6-cb1e-4f63-ad63-962750596484 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:28.466229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11f0a6-cb1e-4f63-ad63-962750596484 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:28.466725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 99/195] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:28.467944 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace20d26-bbe0-4353-84be-c1a9e3f03b24 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:28.472208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11f0a6-cb1e-4f63-ad63-962750596484 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:28.473728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11f0a6-cb1e-4f63-ad63-962750596484 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:28.498512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:28.557313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 97/196] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:16:28 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:28.558430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11f0a6-cb1e-4f63-ad63-962750596484 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:28.567040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace20d26-bbe0-4353-84be-c1a9e3f03b24 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:28.624082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdcd9e-f022-425f-a3b8-6c9e5b2afa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:28.625302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdcd9e-f022-425f-a3b8-6c9e5b2afa73 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:28.626111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdcd9e-f022-425f-a3b8-6c9e5b2afa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:28.626403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdcd9e-f022-425f-a3b8-6c9e5b2afa73 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:28.628106 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdcd9e-f022-425f-a3b8-6c9e5b2afa73 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:28.630781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdcd9e-f022-425f-a3b8-6c9e5b2afa73 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:28.632870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 100/197] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:28 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:28.638713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdcd9e-f022-425f-a3b8-6c9e5b2afa73 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:28.677529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11f0a6-cb1e-4f63-ad63-962750596484 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:28.715623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7df68-09bf-46ef-ad1e-300a108e5e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:28.716419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7df68-09bf-46ef-ad1e-300a108e5e77 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:28.716708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7df68-09bf-46ef-ad1e-300a108e5e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:28.716990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7df68-09bf-46ef-ad1e-300a108e5e77 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:28.719120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7df68-09bf-46ef-ad1e-300a108e5e77 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:28.721817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7df68-09bf-46ef-ad1e-300a108e5e77 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:28.723635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 98/198] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:28 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:28.730461 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7df68-09bf-46ef-ad1e-300a108e5e77 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:28.749571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:28.755744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdcd9e-f022-425f-a3b8-6c9e5b2afa73 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:28.797209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191cb6c-250f-43bb-b7d6-db282ebc3986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:28.798026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191cb6c-250f-43bb-b7d6-db282ebc3986 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:28.798516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:28.798709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191cb6c-250f-43bb-b7d6-db282ebc3986 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:28.798995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191cb6c-250f-43bb-b7d6-db282ebc3986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:28.800126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191cb6c-250f-43bb-b7d6-db282ebc3986 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=7998a56b1396402ab848761300b3785e, user.id=2fa587ae4e84494da315773eef05e33c) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:28.800448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191cb6c-250f-43bb-b7d6-db282ebc3986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:28.802119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191cb6c-250f-43bb-b7d6-db282ebc3986 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:28.810404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 101/199] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:28 2022] GET /identity/v3/role_assignments?user.id=2fa587ae4e84494da315773eef05e33c&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=bb0756f8af694633b9743a958eb46e7a => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:28.864457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7df68-09bf-46ef-ad1e-300a108e5e77 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:28.905967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c657106-3df0-4efd-817f-93ad2b9cf92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:28.906645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c657106-3df0-4efd-817f-93ad2b9cf92f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:28.906932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c657106-3df0-4efd-817f-93ad2b9cf92f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:28.907213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c657106-3df0-4efd-817f-93ad2b9cf92f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:28.908060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c657106-3df0-4efd-817f-93ad2b9cf92f None admin] RBAC: Adding query filter params (role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=7998a56b1396402ab848761300b3785e, user.id=2fa587ae4e84494da315773eef05e33c) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:28.908377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c657106-3df0-4efd-817f-93ad2b9cf92f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:28.909887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c657106-3df0-4efd-817f-93ad2b9cf92f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:28.916424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 99/200] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:28 2022] GET /identity/v3/role_assignments?user.id=2fa587ae4e84494da315773eef05e33c&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:29.601165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf72da-19f0-4ec1-88cb-8879352b4d92 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:29.601729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf72da-19f0-4ec1-88cb-8879352b4d92 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:29.602149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf72da-19f0-4ec1-88cb-8879352b4d92 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:29.603674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 102/201] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:29.609251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32448644-75a4-4a6d-9bf0-ebed0deb0ce8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:29.609734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32448644-75a4-4a6d-9bf0-ebed0deb0ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:29.610163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32448644-75a4-4a6d-9bf0-ebed0deb0ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:29.638141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-32448644-75a4-4a6d-9bf0-ebed0deb0ce8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:29.640113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-32448644-75a4-4a6d-9bf0-ebed0deb0ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:29.729934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 100/202] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:29.774556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf72da-19f0-4ec1-88cb-8879352b4d92 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:29.909913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf72da-19f0-4ec1-88cb-8879352b4d92 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:29.965859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe4aae-302a-44ab-92b1-ad46284de2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:29.966667 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe4aae-302a-44ab-92b1-ad46284de2fe None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:29.966959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe4aae-302a-44ab-92b1-ad46284de2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:29.967271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe4aae-302a-44ab-92b1-ad46284de2fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:29.982755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe4aae-302a-44ab-92b1-ad46284de2fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d2fe4aae-302a-44ab-92b1-ad46284de2fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:29.990638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:29.991787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:29.992790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:29.994852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 103/203] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:30.001314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32448644-75a4-4a6d-9bf0-ebed0deb0ce8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:30.065297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:30.128803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32448644-75a4-4a6d-9bf0-ebed0deb0ce8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:30.172680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:30.173629 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:30.174060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:30.174501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:30.175398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:30.175853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:30.177438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:30.186425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 101/204] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:30.192033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe4aae-302a-44ab-92b1-ad46284de2fe None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:30.223920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:30.325099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe4aae-302a-44ab-92b1-ad46284de2fe None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:30.372174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345f72c-695a-4064-baad-ca36333d0050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:30.373045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345f72c-695a-4064-baad-ca36333d0050 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:30.373357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345f72c-695a-4064-baad-ca36333d0050 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:30.373661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345f72c-695a-4064-baad-ca36333d0050 None admin] PATH_INFO: `/v3/users/2d6ad4bad81247fbbdb2169047f10eb9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:30.377809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345f72c-695a-4064-baad-ca36333d0050 None admin] RBAC: Authorizing `identity:get_user(user_id=2d6ad4bad81247fbbdb2169047f10eb9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:30.379264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345f72c-695a-4064-baad-ca36333d0050 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:30.380880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 104/205] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:30 2022] GET /identity/v3/users/2d6ad4bad81247fbbdb2169047f10eb9 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:30.386459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:30.517311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50f5e3-0728-486d-a962-33e58008c479 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:30.519236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:30.565036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cb6b0-96f3-4fd3-8940-dcbad52fc923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:30.565719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cb6b0-96f3-4fd3-8940-dcbad52fc923 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:30.566265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cb6b0-96f3-4fd3-8940-dcbad52fc923 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:30.566930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cb6b0-96f3-4fd3-8940-dcbad52fc923 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:30.656481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cb6b0-96f3-4fd3-8940-dcbad52fc923 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:30.657981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cb6b0-96f3-4fd3-8940-dcbad52fc923 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:30.772847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 102/206] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:30.782557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345f72c-695a-4064-baad-ca36333d0050 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:30.876611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:30.886036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345f72c-695a-4064-baad-ca36333d0050 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:30.920017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3625e-e33d-4b08-bf2e-bbae38f1b0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:30.921582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3625e-e33d-4b08-bf2e-bbae38f1b0bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:30.921963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3625e-e33d-4b08-bf2e-bbae38f1b0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:30.922357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3625e-e33d-4b08-bf2e-bbae38f1b0bb None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:30.923664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3625e-e33d-4b08-bf2e-bbae38f1b0bb None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:30.925256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3625e-e33d-4b08-bf2e-bbae38f1b0bb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:30.927142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 105/207] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:30 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:30.933062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cb6b0-96f3-4fd3-8940-dcbad52fc923 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:30.937401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:31.099086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cb6b0-96f3-4fd3-8940-dcbad52fc923 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:31.153914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fefa7-bb5e-42dc-8cdb-39f0a944fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:31.154568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fefa7-bb5e-42dc-8cdb-39f0a944fe3e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.155387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fefa7-bb5e-42dc-8cdb-39f0a944fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.155664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fefa7-bb5e-42dc-8cdb-39f0a944fe3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.156487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fefa7-bb5e-42dc-8cdb-39f0a944fe3e None admin] RBAC: Adding query filter params (role.id=86bf554ccc4b46ee91870389659035e3, scope.project.id=cd271be4aaeb4efab5ef86157a14122b, user.id=2d6ad4bad81247fbbdb2169047f10eb9) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:31.156893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fefa7-bb5e-42dc-8cdb-39f0a944fe3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:31.158411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fefa7-bb5e-42dc-8cdb-39f0a944fe3e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:31.169806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 103/208] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:30 2022] GET /identity/v3/role_assignments?user.id=2d6ad4bad81247fbbdb2169047f10eb9&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=86bf554ccc4b46ee91870389659035e3 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:31.482633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b49b5-4c83-4b07-8ec1-348f201a3714 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.483512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b49b5-4c83-4b07-8ec1-348f201a3714 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.483703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b49b5-4c83-4b07-8ec1-348f201a3714 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.485246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 106/209] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:31.490343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08647-6894-4b48-b434-a31570ba7330 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.490636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08647-6894-4b48-b434-a31570ba7330 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.490913 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08647-6894-4b48-b434-a31570ba7330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.508470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-96c08647-6894-4b48-b434-a31570ba7330 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:31.509693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-96c08647-6894-4b48-b434-a31570ba7330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:31.567296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ac850-d98e-4626-9eda-35e34f9e9155 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.567867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ac850-d98e-4626-9eda-35e34f9e9155 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.568378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ac850-d98e-4626-9eda-35e34f9e9155 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.569888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 107/210] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:31.576218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287711c-3a2d-4527-8b36-89aa6ee322b3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.577304 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287711c-3a2d-4527-8b36-89aa6ee322b3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.577794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287711c-3a2d-4527-8b36-89aa6ee322b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.580412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:31.594783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 104/211] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:31.597149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2287711c-3a2d-4527-8b36-89aa6ee322b3 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:31.602589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2287711c-3a2d-4527-8b36-89aa6ee322b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:31.623027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c08647-6894-4b48-b434-a31570ba7330 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:31.685228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 108/212] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:31.720845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287711c-3a2d-4527-8b36-89aa6ee322b3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:31.757631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c08647-6894-4b48-b434-a31570ba7330 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:31.799324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7958d3b-e025-4785-9944-4e6f433909d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:31.800109 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7958d3b-e025-4785-9944-4e6f433909d3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.800391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7958d3b-e025-4785-9944-4e6f433909d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.800659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7958d3b-e025-4785-9944-4e6f433909d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.810917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7958d3b-e025-4785-9944-4e6f433909d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e7958d3b-e025-4785-9944-4e6f433909d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:31.819528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:31.820550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 105/213] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:31.828788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384d214-480a-43f3-82c1-4730785c7071 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.829724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384d214-480a-43f3-82c1-4730785c7071 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.829724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384d214-480a-43f3-82c1-4730785c7071 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.829724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 106/214] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:31.829724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384d214-480a-43f3-82c1-4730785c7071 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:31.837289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:31.849668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287711c-3a2d-4527-8b36-89aa6ee322b3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:31.888810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1748a-692c-4b67-a40a-e5c9a5908345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:31.889684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:31.889838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1748a-692c-4b67-a40a-e5c9a5908345 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.890354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1748a-692c-4b67-a40a-e5c9a5908345 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.890354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1748a-692c-4b67-a40a-e5c9a5908345 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.891512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1748a-692c-4b67-a40a-e5c9a5908345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:31.893011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1748a-692c-4b67-a40a-e5c9a5908345 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:31.894717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 109/215] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:31.898504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a525e5b-7629-4c01-93a1-02dd8ab17f8a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.898959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a525e5b-7629-4c01-93a1-02dd8ab17f8a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.899433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a525e5b-7629-4c01-93a1-02dd8ab17f8a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.903737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 110/216] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:31.903737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4d3a0-bcf3-4841-aaac-cb4b1e28478a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.903737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4d3a0-bcf3-4841-aaac-cb4b1e28478a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.903737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4d3a0-bcf3-4841-aaac-cb4b1e28478a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.919867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-71a4d3a0-bcf3-4841-aaac-cb4b1e28478a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:31.921645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-71a4d3a0-bcf3-4841-aaac-cb4b1e28478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:31.938078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384d214-480a-43f3-82c1-4730785c7071 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:31.970706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:31.971402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.971941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.972218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:31.972961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:31.973305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:31.974717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:31.982718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 107/217] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:31.982718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:31.992349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 111/218] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:31.998561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221623b-b235-49e9-9e54-1aeedea06f9e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:31.998858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221623b-b235-49e9-9e54-1aeedea06f9e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:31.999253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221623b-b235-49e9-9e54-1aeedea06f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.016563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d221623b-b235-49e9-9e54-1aeedea06f9e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:32.017708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d221623b-b235-49e9-9e54-1aeedea06f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:32.081661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781efa4-2176-49e3-a399-9c205ee61284 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.086413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 112/219] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:32.087099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221623b-b235-49e9-9e54-1aeedea06f9e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.100909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:32.105799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:32.117628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238b16b-c275-440b-a028-bdac76c84829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.118294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b16b-c275-440b-a028-bdac76c84829 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.118572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b16b-c275-440b-a028-bdac76c84829 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.118845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b16b-c275-440b-a028-bdac76c84829 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.119544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238b16b-c275-440b-a028-bdac76c84829 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:32.120976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238b16b-c275-440b-a028-bdac76c84829 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:32.189907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 108/220] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:16:31 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:32.189907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238b16b-c275-440b-a028-bdac76c84829 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.205415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221623b-b235-49e9-9e54-1aeedea06f9e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.240204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb64a16-c0bf-4bf1-837c-963487b6acd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.241130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb64a16-c0bf-4bf1-837c-963487b6acd4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.241412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb64a16-c0bf-4bf1-837c-963487b6acd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.241691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb64a16-c0bf-4bf1-837c-963487b6acd4 None admin] PATH_INFO: `/v3/users/2fa587ae4e84494da315773eef05e33c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.242965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb64a16-c0bf-4bf1-837c-963487b6acd4 None admin] RBAC: Authorizing `identity:get_user(user_id=2fa587ae4e84494da315773eef05e33c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:32.244488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb64a16-c0bf-4bf1-837c-963487b6acd4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:32.250875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 113/221] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:32 2022] GET /identity/v3/users/2fa587ae4e84494da315773eef05e33c => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:32.250875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb64a16-c0bf-4bf1-837c-963487b6acd4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.311111 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238b16b-c275-440b-a028-bdac76c84829 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.348711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6683f-da85-4ef8-8326-9b8a09af57f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.349555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6683f-da85-4ef8-8326-9b8a09af57f1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.349841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6683f-da85-4ef8-8326-9b8a09af57f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.350118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6683f-da85-4ef8-8326-9b8a09af57f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.352533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:32.362628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6683f-da85-4ef8-8326-9b8a09af57f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:32.369227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb64a16-c0bf-4bf1-837c-963487b6acd4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-dcd6683f-da85-4ef8-8326-9b8a09af57f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:32.370438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:32.371329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:32.372132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 109/222] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:32.372758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6683f-da85-4ef8-8326-9b8a09af57f1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.415882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:32.418645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc6720-2a7e-4b32-a4f8-addcd62db974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.419460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc6720-2a7e-4b32-a4f8-addcd62db974 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.419748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc6720-2a7e-4b32-a4f8-addcd62db974 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.420026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc6720-2a7e-4b32-a4f8-addcd62db974 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.434358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc6720-2a7e-4b32-a4f8-addcd62db974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-53dc6720-2a7e-4b32-a4f8-addcd62db974 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:32.446916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:32.447824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:32.449040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 114/223] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:32.452957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc6720-2a7e-4b32-a4f8-addcd62db974 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.519121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6683f-da85-4ef8-8326-9b8a09af57f1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.558319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9bce5-4451-4a64-9d56-0355a98a9c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.559192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9bce5-4451-4a64-9d56-0355a98a9c74 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.559472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9bce5-4451-4a64-9d56-0355a98a9c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.559755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9bce5-4451-4a64-9d56-0355a98a9c74 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.561002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9bce5-4451-4a64-9d56-0355a98a9c74 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:32.562560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9bce5-4451-4a64-9d56-0355a98a9c74 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:32.564196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 110/224] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:32 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:32.568759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9bce5-4451-4a64-9d56-0355a98a9c74 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.582231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc6720-2a7e-4b32-a4f8-addcd62db974 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.617493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.618223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.618517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.618797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.619609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:32.619924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:32.621369 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:32.630522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 115/225] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:32.631382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.645455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:32.684510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9bce5-4451-4a64-9d56-0355a98a9c74 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.686647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:32.729125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.729827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.730113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.730391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.731316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:32.731628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:32.734229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:32.741403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de4677-a706-444c-bb44-ca40f09f4408 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.745013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 111/226] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:32.749913 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.783652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab79c85-3b12-43a9-91c0-d19f095b6072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.784854 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab79c85-3b12-43a9-91c0-d19f095b6072 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.785460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab79c85-3b12-43a9-91c0-d19f095b6072 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.785915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab79c85-3b12-43a9-91c0-d19f095b6072 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e/users/2fa587ae4e84494da315773eef05e33c/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.788351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab79c85-3b12-43a9-91c0-d19f095b6072 None admin] RBAC: Authorizing `identity:create_grant(project_id=7998a56b1396402ab848761300b3785e, user_id=2fa587ae4e84494da315773eef05e33c, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:32.790148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab79c85-3b12-43a9-91c0-d19f095b6072 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:32.868176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 116/227] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:32 2022] PUT /identity/v3/projects/7998a56b1396402ab848761300b3785e/users/2fa587ae4e84494da315773eef05e33c/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:32.878120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab79c85-3b12-43a9-91c0-d19f095b6072 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:32.884022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882357c1-47b9-4b61-b7e2-7dd3b3a0e836 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:32.962235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ee65a-b17d-481e-9d55-0be8fa6a6b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:32.963134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ee65a-b17d-481e-9d55-0be8fa6a6b67 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:32.963417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ee65a-b17d-481e-9d55-0be8fa6a6b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:32.963690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ee65a-b17d-481e-9d55-0be8fa6a6b67 None admin] PATH_INFO: `/v3/users/3c27f9cba4324127a8697c6ee62a0320` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:32.996100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ee65a-b17d-481e-9d55-0be8fa6a6b67 None admin] RBAC: Authorizing `identity:get_user(user_id=3c27f9cba4324127a8697c6ee62a0320)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:32.997654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ee65a-b17d-481e-9d55-0be8fa6a6b67 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:33.000189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 112/228] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:32 2022] GET /identity/v3/users/3c27f9cba4324127a8697c6ee62a0320 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:33.006330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ee65a-b17d-481e-9d55-0be8fa6a6b67 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:33.017487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:33.059073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:33.096406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab79c85-3b12-43a9-91c0-d19f095b6072 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:33.140477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2b039-adc5-45f5-bd30-1bc008cfb74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:33.141699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2b039-adc5-45f5-bd30-1bc008cfb74c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:33.142171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2b039-adc5-45f5-bd30-1bc008cfb74c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:33.142643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2b039-adc5-45f5-bd30-1bc008cfb74c None admin] PATH_INFO: `/v3/users/2fa587ae4e84494da315773eef05e33c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:33.144157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2b039-adc5-45f5-bd30-1bc008cfb74c None admin] RBAC: Authorizing `identity:get_user(user_id=2fa587ae4e84494da315773eef05e33c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:33.145820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2b039-adc5-45f5-bd30-1bc008cfb74c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:33.147640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 117/229] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:32 2022] GET /identity/v3/users/2fa587ae4e84494da315773eef05e33c => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:33.153013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2b039-adc5-45f5-bd30-1bc008cfb74c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:33.164699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ee65a-b17d-481e-9d55-0be8fa6a6b67 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:33.206319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd089-7cff-4403-ac56-a03a1b14fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:33.207107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd089-7cff-4403-ac56-a03a1b14fa6a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:33.207456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd089-7cff-4403-ac56-a03a1b14fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:33.207800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdd089-7cff-4403-ac56-a03a1b14fa6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:33.261787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2b039-adc5-45f5-bd30-1bc008cfb74c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:33.300354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a563f-1970-4bbb-a480-8673615414f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:33.303006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a563f-1970-4bbb-a480-8673615414f3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:33.304166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a563f-1970-4bbb-a480-8673615414f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:33.305328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a563f-1970-4bbb-a480-8673615414f3 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:33.308628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a563f-1970-4bbb-a480-8673615414f3 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:33.310884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a563f-1970-4bbb-a480-8673615414f3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:33.316189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 118/230] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:33 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:33.322466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdd089-7cff-4403-ac56-a03a1b14fa6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:33.324150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdd089-7cff-4403-ac56-a03a1b14fa6a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:33.327943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a563f-1970-4bbb-a480-8673615414f3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:33.338425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:33.377455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:33.464918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 113/231] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:33.472687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd089-7cff-4403-ac56-a03a1b14fa6a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:33.513401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a563f-1970-4bbb-a480-8673615414f3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:33.553001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709b756-6ed1-4663-a9bb-9b847102be83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:33.553915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709b756-6ed1-4663-a9bb-9b847102be83 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:33.554196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709b756-6ed1-4663-a9bb-9b847102be83 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:33.554475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709b756-6ed1-4663-a9bb-9b847102be83 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e/users/2fa587ae4e84494da315773eef05e33c/roles/fc5d6a08fc2048f19c14f691e35cd41c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:33.559212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709b756-6ed1-4663-a9bb-9b847102be83 None admin] RBAC: Authorizing `identity:create_grant(project_id=7998a56b1396402ab848761300b3785e, user_id=2fa587ae4e84494da315773eef05e33c, role_id=fc5d6a08fc2048f19c14f691e35cd41c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:33.560688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709b756-6ed1-4663-a9bb-9b847102be83 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:33.637813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 119/232] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:33 2022] PUT /identity/v3/projects/7998a56b1396402ab848761300b3785e/users/2fa587ae4e84494da315773eef05e33c/roles/fc5d6a08fc2048f19c14f691e35cd41c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:16:33.655403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdd089-7cff-4403-ac56-a03a1b14fa6a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:33.690044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:33.710597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25bf28-e63d-495e-9c4b-8877debdd6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:33.711485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25bf28-e63d-495e-9c4b-8877debdd6bb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:33.711774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25bf28-e63d-495e-9c4b-8877debdd6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:33.712051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25bf28-e63d-495e-9c4b-8877debdd6bb None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:33.713317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25bf28-e63d-495e-9c4b-8877debdd6bb None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:33.715502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25bf28-e63d-495e-9c4b-8877debdd6bb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:33.721369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 114/233] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:33 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:33.728158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709b756-6ed1-4663-a9bb-9b847102be83 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:33.829397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:33.878054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709b756-6ed1-4663-a9bb-9b847102be83 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:33.927552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9677666-3238-4aee-802c-eb3de52a7809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:33.928210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9677666-3238-4aee-802c-eb3de52a7809 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:33.928485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9677666-3238-4aee-802c-eb3de52a7809 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:33.928764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9677666-3238-4aee-802c-eb3de52a7809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:33.929637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9677666-3238-4aee-802c-eb3de52a7809 None admin] RBAC: Adding query filter params (role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=7998a56b1396402ab848761300b3785e, user.id=3c27f9cba4324127a8697c6ee62a0320) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:33.929957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9677666-3238-4aee-802c-eb3de52a7809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:33.931484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9677666-3238-4aee-802c-eb3de52a7809 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:33.938096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 120/234] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:33 2022] GET /identity/v3/role_assignments?user.id=3c27f9cba4324127a8697c6ee62a0320&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:34.899046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ace802-c020-4ff9-86d4-b1fb20617a8d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:34.899565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ace802-c020-4ff9-86d4-b1fb20617a8d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:34.899673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ace802-c020-4ff9-86d4-b1fb20617a8d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:34.901151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 115/235] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:34.918648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fe369-b7ba-45cd-b5c0-b2d42030de5c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:34.919608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fe369-b7ba-45cd-b5c0-b2d42030de5c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:34.920091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fe369-b7ba-45cd-b5c0-b2d42030de5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:34.939242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a11fe369-b7ba-45cd-b5c0-b2d42030de5c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:34.940756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a11fe369-b7ba-45cd-b5c0-b2d42030de5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:35.028405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 121/236] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:35.056359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ace802-c020-4ff9-86d4-b1fb20617a8d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:35.097827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412820a-d0cc-41cf-a099-98184fe1efc6 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.098480 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412820a-d0cc-41cf-a099-98184fe1efc6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.098953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412820a-d0cc-41cf-a099-98184fe1efc6 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.100894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 122/237] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:35.119199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732726b-a67a-423f-b6ef-8c1dff00e219 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.119786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732726b-a67a-423f-b6ef-8c1dff00e219 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.120283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732726b-a67a-423f-b6ef-8c1dff00e219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.144637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c732726b-a67a-423f-b6ef-8c1dff00e219 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:35.146380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c732726b-a67a-423f-b6ef-8c1dff00e219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:35.217907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:35.237281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 123/238] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:35.260449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ace802-c020-4ff9-86d4-b1fb20617a8d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:35.304104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732726b-a67a-423f-b6ef-8c1dff00e219 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:35.312293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8c0bf-6e18-4b18-8c6c-5e47bc161e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:35.313093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8c0bf-6e18-4b18-8c6c-5e47bc161e3a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.313398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8c0bf-6e18-4b18-8c6c-5e47bc161e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.313671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8c0bf-6e18-4b18-8c6c-5e47bc161e3a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.331999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8c0bf-6e18-4b18-8c6c-5e47bc161e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-fde8c0bf-6e18-4b18-8c6c-5e47bc161e3a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:35.339675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:35.340548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:35.341357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:35.349607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 116/239] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:35.357605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8c0bf-6e18-4b18-8c6c-5e47bc161e3a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:35.429527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732726b-a67a-423f-b6ef-8c1dff00e219 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:35.471181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc582d-ea57-48ba-b3d2-abcb737f6f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:35.472619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc582d-ea57-48ba-b3d2-abcb737f6f96 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.473121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc582d-ea57-48ba-b3d2-abcb737f6f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.473645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc582d-ea57-48ba-b3d2-abcb737f6f96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.474504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:35.476552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc582d-ea57-48ba-b3d2-abcb737f6f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:35.478777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc582d-ea57-48ba-b3d2-abcb737f6f96 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:35.480990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 124/240] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:35.494092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc582d-ea57-48ba-b3d2-abcb737f6f96 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:35.559120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8c0bf-6e18-4b18-8c6c-5e47bc161e3a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:35.595845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:35.618411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:35.619093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.619374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.619643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.620398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:35.620705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:35.622141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:35.641577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 117/241] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:35.642880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc582d-ea57-48ba-b3d2-abcb737f6f96 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:35.647518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:35.684034 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e99ad-87ba-40c0-aee3-be109397fb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:35.684710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e99ad-87ba-40c0-aee3-be109397fb68 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.684991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e99ad-87ba-40c0-aee3-be109397fb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.685293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e99ad-87ba-40c0-aee3-be109397fb68 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.686543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e99ad-87ba-40c0-aee3-be109397fb68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:35.688000 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e99ad-87ba-40c0-aee3-be109397fb68 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:35.772712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 125/242] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:16:35 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:35.774928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba024-97dc-4d92-9e5c-4dc8a8a7c4cc None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.775287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba024-97dc-4d92-9e5c-4dc8a8a7c4cc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.775639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ba024-97dc-4d92-9e5c-4dc8a8a7c4cc None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.781188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 126/243] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:35.784564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17b1e-daab-4547-acd2-dc20a9777b46 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.784832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17b1e-daab-4547-acd2-dc20a9777b46 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.785132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17b1e-daab-4547-acd2-dc20a9777b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.802274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595bb89-ab2b-49af-b9b8-df1ba2e161a8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:35.826237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-58f17b1e-daab-4547-acd2-dc20a9777b46 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:35.826963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:35.827468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-58f17b1e-daab-4547-acd2-dc20a9777b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:35.853151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ada949-0a93-437f-808b-2b693047e1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:35.854268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ada949-0a93-437f-808b-2b693047e1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:35.854695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ada949-0a93-437f-808b-2b693047e1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:35.855007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ada949-0a93-437f-808b-2b693047e1a8 None admin] PATH_INFO: `/v3/users/7ece33cfe43945929e9db5e5a9f6408a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:35.873652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ada949-0a93-437f-808b-2b693047e1a8 None admin] RBAC: Authorizing `identity:get_user(user_id=7ece33cfe43945929e9db5e5a9f6408a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:35.875283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ada949-0a93-437f-808b-2b693047e1a8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:35.877176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 118/244] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:35 2022] GET /identity/v3/users/7ece33cfe43945929e9db5e5a9f6408a => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:35.883079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ada949-0a93-437f-808b-2b693047e1a8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.011024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ada949-0a93-437f-808b-2b693047e1a8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:36.034791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 127/245] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:36.066162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6b82c-efd5-4b12-915f-cce05a72125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:36.066828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6b82c-efd5-4b12-915f-cce05a72125e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.067115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6b82c-efd5-4b12-915f-cce05a72125e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.067240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17b1e-daab-4547-acd2-dc20a9777b46 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.067399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6b82c-efd5-4b12-915f-cce05a72125e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.114763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:36.124679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:36.169090 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6b82c-efd5-4b12-915f-cce05a72125e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:36.171121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6b82c-efd5-4b12-915f-cce05a72125e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:36.188771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17b1e-daab-4547-acd2-dc20a9777b46 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:36.237393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e75dc3-6e0e-4b6a-a5da-747112486c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:36.240234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e75dc3-6e0e-4b6a-a5da-747112486c2f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.240234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e75dc3-6e0e-4b6a-a5da-747112486c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.240234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e75dc3-6e0e-4b6a-a5da-747112486c2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.253841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e75dc3-6e0e-4b6a-a5da-747112486c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-62e75dc3-6e0e-4b6a-a5da-747112486c2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:36.263476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:36.264976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:36.265825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:36.266904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 128/246] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:16:36.267772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02d318-d28e-4283-a1ec-1153d15bda1e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.268063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02d318-d28e-4283-a1ec-1153d15bda1e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.268336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02d318-d28e-4283-a1ec-1153d15bda1e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.270259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 129/247] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:36.273997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02d318-d28e-4283-a1ec-1153d15bda1e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.309601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 119/248] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:36.311159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beab72f-7e7d-49f7-8743-c3d4f27a0552 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.311462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beab72f-7e7d-49f7-8743-c3d4f27a0552 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.311719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beab72f-7e7d-49f7-8743-c3d4f27a0552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.334604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2beab72f-7e7d-49f7-8743-c3d4f27a0552 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:36.336584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2beab72f-7e7d-49f7-8743-c3d4f27a0552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:36.410365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:36.412491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c02d318-d28e-4283-a1ec-1153d15bda1e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:36.418203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 120/249] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:36.418663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:36.419897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beab72f-7e7d-49f7-8743-c3d4f27a0552 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.461920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:36.462613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.462897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.463175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.464225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:36.464543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:36.466956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:36.478877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 130/250] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:36.479583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.550584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beab72f-7e7d-49f7-8743-c3d4f27a0552 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:36.593203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e2b20-d0af-4dde-84dd-e90f803edf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:36.593996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e2b20-d0af-4dde-84dd-e90f803edf4f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.594276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e2b20-d0af-4dde-84dd-e90f803edf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.594550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e2b20-d0af-4dde-84dd-e90f803edf4f None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.596462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e2b20-d0af-4dde-84dd-e90f803edf4f None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:36.598212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e2b20-d0af-4dde-84dd-e90f803edf4f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:36.600172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 121/251] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:36 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:36.600707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e2b20-d0af-4dde-84dd-e90f803edf4f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.620627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33f156-6d16-4459-a78b-fee6cdb9b933 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:36.664484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac1327-a53e-4650-8fab-bc39b3fd29dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:36.665883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac1327-a53e-4650-8fab-bc39b3fd29dd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.666170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac1327-a53e-4650-8fab-bc39b3fd29dd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.666452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac1327-a53e-4650-8fab-bc39b3fd29dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.673466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:36.680737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac1327-a53e-4650-8fab-bc39b3fd29dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-aaac1327-a53e-4650-8fab-bc39b3fd29dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:36.691236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:36.692154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:36.692927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 131/252] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:36.693603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac1327-a53e-4650-8fab-bc39b3fd29dd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.706035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e2b20-d0af-4dde-84dd-e90f803edf4f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:36.725308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:36.742834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40231858-c553-458f-b211-f5d99abf636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:36.743971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40231858-c553-458f-b211-f5d99abf636c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.744416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40231858-c553-458f-b211-f5d99abf636c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.744945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40231858-c553-458f-b211-f5d99abf636c None admin] PATH_INFO: `/v3/users/2fa587ae4e84494da315773eef05e33c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.747258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40231858-c553-458f-b211-f5d99abf636c None admin] RBAC: Authorizing `identity:get_user(user_id=2fa587ae4e84494da315773eef05e33c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:36.750112 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40231858-c553-458f-b211-f5d99abf636c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:36.752394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 122/253] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:36 2022] GET /identity/v3/users/2fa587ae4e84494da315773eef05e33c => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:36.753785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbb4d2-6564-4291-8a04-08c188e83cec None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.754239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbb4d2-6564-4291-8a04-08c188e83cec None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.754657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbb4d2-6564-4291-8a04-08c188e83cec None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.756033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 123/254] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:36.757605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbb4d2-6564-4291-8a04-08c188e83cec None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.830091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac1327-a53e-4650-8fab-bc39b3fd29dd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:36.865934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:36.866813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.867245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.867665 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.868646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] RBAC: Adding query filter params (role.id=86bf554ccc4b46ee91870389659035e3, scope.project.id=7998a56b1396402ab848761300b3785e, user.id=7ece33cfe43945929e9db5e5a9f6408a) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:36.869144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:36.870731 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:36.877187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 132/255] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:36 2022] GET /identity/v3/role_assignments?user.id=7ece33cfe43945929e9db5e5a9f6408a&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=86bf554ccc4b46ee91870389659035e3 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:36.878242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:36.905549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbb4d2-6564-4291-8a04-08c188e83cec None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:36.958822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d3c8e-fc82-46e1-b9df-17d5b2a31be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:36.959676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d3c8e-fc82-46e1-b9df-17d5b2a31be4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.960051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d3c8e-fc82-46e1-b9df-17d5b2a31be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.960425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d3c8e-fc82-46e1-b9df-17d5b2a31be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:36.961366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d3c8e-fc82-46e1-b9df-17d5b2a31be4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:36.961799 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d3c8e-fc82-46e1-b9df-17d5b2a31be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:36.963632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d3c8e-fc82-46e1-b9df-17d5b2a31be4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:36.968102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:36.976066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 124/256] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:36.980442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1105f2-426e-49dd-a86d-73a278c0c0be None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:36.980442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1105f2-426e-49dd-a86d-73a278c0c0be None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:36.980442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1105f2-426e-49dd-a86d-73a278c0c0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:37.014702 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6d1105f2-426e-49dd-a86d-73a278c0c0be None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:37.015934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6d1105f2-426e-49dd-a86d-73a278c0c0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:37.059757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:37.071532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ccac2-0751-4009-ad70-d53c8b0eab3f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:37.117416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e42c8-8466-4f53-80d6-53fd3e60d95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:37.119624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e42c8-8466-4f53-80d6-53fd3e60d95c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:37.120945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e42c8-8466-4f53-80d6-53fd3e60d95c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:37.121453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e42c8-8466-4f53-80d6-53fd3e60d95c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:37.159658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 125/257] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:37.161341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1105f2-426e-49dd-a86d-73a278c0c0be None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:37.232012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e42c8-8466-4f53-80d6-53fd3e60d95c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:37.234456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e42c8-8466-4f53-80d6-53fd3e60d95c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:37.373253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1105f2-426e-49dd-a86d-73a278c0c0be None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:37.389455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:37.419743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 133/258] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:37.421292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e42c8-8466-4f53-80d6-53fd3e60d95c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:37.437477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:37.445942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459fbf-7322-4249-9d5a-ce66967def55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:37.446825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459fbf-7322-4249-9d5a-ce66967def55 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:37.447104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459fbf-7322-4249-9d5a-ce66967def55 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:37.447377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459fbf-7322-4249-9d5a-ce66967def55 None admin] PATH_INFO: `/v3/users/2fa587ae4e84494da315773eef05e33c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:37.450061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5459fbf-7322-4249-9d5a-ce66967def55 None admin] RBAC: Authorizing `identity:get_user(user_id=2fa587ae4e84494da315773eef05e33c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:37.452807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5459fbf-7322-4249-9d5a-ce66967def55 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:37.454669 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 126/259] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:37 2022] GET /identity/v3/users/2fa587ae4e84494da315773eef05e33c => generated 283 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:37.455263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459fbf-7322-4249-9d5a-ce66967def55 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:37.597027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e42c8-8466-4f53-80d6-53fd3e60d95c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:37.612135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459fbf-7322-4249-9d5a-ce66967def55 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:37.632921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cab706-15f1-4f53-b5c2-4e8198dd4934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:37.633671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cab706-15f1-4f53-b5c2-4e8198dd4934 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:37.633954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cab706-15f1-4f53-b5c2-4e8198dd4934 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:37.634230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cab706-15f1-4f53-b5c2-4e8198dd4934 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:37.647050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cab706-15f1-4f53-b5c2-4e8198dd4934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:37.654877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cc9fd-2ceb-4dfd-90fe-7e76183aa8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:37.655748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cc9fd-2ceb-4dfd-90fe-7e76183aa8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:37.656033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cc9fd-2ceb-4dfd-90fe-7e76183aa8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:37.656314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cc9fd-2ceb-4dfd-90fe-7e76183aa8f5 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:37.657537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cc9fd-2ceb-4dfd-90fe-7e76183aa8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:37.659011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cc9fd-2ceb-4dfd-90fe-7e76183aa8f5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:37.660710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 127/260] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:37 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:37.661496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cc9fd-2ceb-4dfd-90fe-7e76183aa8f5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-73cab706-15f1-4f53-b5c2-4e8198dd4934 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:37.663203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:37.664056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:37.664830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:37.667156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 134/261] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:37.667959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cab706-15f1-4f53-b5c2-4e8198dd4934 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:37.737458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:37.772424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cc9fd-2ceb-4dfd-90fe-7e76183aa8f5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:37.782509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:37.810188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14ae81-9da2-41fa-8858-50c85331780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:37.811394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ae81-9da2-41fa-8858-50c85331780b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:37.811686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ae81-9da2-41fa-8858-50c85331780b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:37.811964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ae81-9da2-41fa-8858-50c85331780b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:37.886340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cab706-15f1-4f53-b5c2-4e8198dd4934 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:37.924623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14ae81-9da2-41fa-8858-50c85331780b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:37.927091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14ae81-9da2-41fa-8858-50c85331780b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:37.962807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:37.963509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:37.963793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:37.964183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:37.971464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=7998a56b1396402ab848761300b3785e, user.id=2fa587ae4e84494da315773eef05e33c) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:37.971464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:37.971464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:37.985938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 135/262] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:37 2022] GET /identity/v3/role_assignments?user.id=2fa587ae4e84494da315773eef05e33c&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=bb0756f8af694633b9743a958eb46e7a => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:37.987214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:37.987821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:38.028222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 128/263] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:38.034140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14ae81-9da2-41fa-8858-50c85331780b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:38.157616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14ae81-9da2-41fa-8858-50c85331780b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:38.194157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9b228-9805-45ca-968b-8347e4e13d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:38.195244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9b228-9805-45ca-968b-8347e4e13d4a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:38.195624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9b228-9805-45ca-968b-8347e4e13d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:38.196000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9b228-9805-45ca-968b-8347e4e13d4a None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:38.197358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9b228-9805-45ca-968b-8347e4e13d4a None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:38.198948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9b228-9805-45ca-968b-8347e4e13d4a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:38.200831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 129/264] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:38 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:38.206871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9b228-9805-45ca-968b-8347e4e13d4a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:38.247180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97afe6-1281-4ab2-ad90-00fe1e8bb575 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:38.257499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:38.270941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:38.307736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:38.308405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:38.308687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:38.309142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:38.309973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:38.310344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:38.311886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:38.323653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9b228-9805-45ca-968b-8347e4e13d4a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:38.324214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 136/265] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:38.331298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:38.361993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:38.362659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:38.362949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:38.363231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:38.364070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] RBAC: Adding query filter params (role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=7998a56b1396402ab848761300b3785e, user.id=2fa587ae4e84494da315773eef05e33c) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:38.364386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:38.365961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:38.374170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 130/266] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:38 2022] GET /identity/v3/role_assignments?user.id=2fa587ae4e84494da315773eef05e33c&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:38.446847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a34691-4771-4f45-9417-6796b6b5dffc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:38.486881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60f24-d7ad-433e-89e8-9d3a90d8fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:38.487771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60f24-d7ad-433e-89e8-9d3a90d8fba1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:38.488051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60f24-d7ad-433e-89e8-9d3a90d8fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:38.488328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d60f24-d7ad-433e-89e8-9d3a90d8fba1 None admin] PATH_INFO: `/v3/users/3c27f9cba4324127a8697c6ee62a0320` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:38.489637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60f24-d7ad-433e-89e8-9d3a90d8fba1 None admin] RBAC: Authorizing `identity:get_user(user_id=3c27f9cba4324127a8697c6ee62a0320)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:38.491117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d60f24-d7ad-433e-89e8-9d3a90d8fba1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:38.497238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 137/267] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:38 2022] GET /identity/v3/users/3c27f9cba4324127a8697c6ee62a0320 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:38.498858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:38.621415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:38.627306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824ab92-c556-4343-897a-3744f8a5969e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:38.670231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebe9da-778f-4f9c-81f7-f3d4770919ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:38.671094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebe9da-778f-4f9c-81f7-f3d4770919ba None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:38.671388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebe9da-778f-4f9c-81f7-f3d4770919ba None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:38.671642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebe9da-778f-4f9c-81f7-f3d4770919ba None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:38.672825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebe9da-778f-4f9c-81f7-f3d4770919ba None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:38.674361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebe9da-778f-4f9c-81f7-f3d4770919ba None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:38.676203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 131/268] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:38 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:38.681908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60f24-d7ad-433e-89e8-9d3a90d8fba1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:38.726549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:38.790437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d60f24-d7ad-433e-89e8-9d3a90d8fba1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:38.827485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b18e8-37a6-449f-82e7-739b769aad21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:38.828371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b18e8-37a6-449f-82e7-739b769aad21 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:38.828650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b18e8-37a6-449f-82e7-739b769aad21 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:38.828929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b18e8-37a6-449f-82e7-739b769aad21 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e/users/3c27f9cba4324127a8697c6ee62a0320/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:38.830945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b18e8-37a6-449f-82e7-739b769aad21 None admin] RBAC: Authorizing `identity:create_grant(project_id=7998a56b1396402ab848761300b3785e, user_id=3c27f9cba4324127a8697c6ee62a0320, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:38.832426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b18e8-37a6-449f-82e7-739b769aad21 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:38.897139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 138/269] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:38 2022] PUT /identity/v3/projects/7998a56b1396402ab848761300b3785e/users/3c27f9cba4324127a8697c6ee62a0320/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:38.974930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ec03-5224-4d24-a1e7-d20e8a3d4132 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:38.975218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ec03-5224-4d24-a1e7-d20e8a3d4132 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:38.975736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ec03-5224-4d24-a1e7-d20e8a3d4132 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:38.977474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 132/270] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:38.984080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5476d-27f2-4558-bc20-51975d7acd6c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:38.984686 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5476d-27f2-4558-bc20-51975d7acd6c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:38.985193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5476d-27f2-4558-bc20-51975d7acd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.004015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-39a5476d-27f2-4558-bc20-51975d7acd6c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:39.005698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-39a5476d-27f2-4558-bc20-51975d7acd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:39.083841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 139/271] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:39.107213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ec03-5224-4d24-a1e7-d20e8a3d4132 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:39.120079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c8940-c5eb-4b78-a4a7-1ad99feacad1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.120634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c8940-c5eb-4b78-a4a7-1ad99feacad1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.121057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c8940-c5eb-4b78-a4a7-1ad99feacad1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.122912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 140/272] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:39.128320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344246-76fb-4475-a906-137683371233 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.128651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344246-76fb-4475-a906-137683371233 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.128998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344246-76fb-4475-a906-137683371233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.149594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-25344246-76fb-4475-a906-137683371233 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:39.150877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-25344246-76fb-4475-a906-137683371233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:39.183715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:39.225425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ec03-5224-4d24-a1e7-d20e8a3d4132 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:39.230480 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 141/273] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:39.256237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25344246-76fb-4475-a906-137683371233 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:39.263440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0b571-202a-415f-a822-197b9f69aa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:39.264261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0b571-202a-415f-a822-197b9f69aa6c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.264570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0b571-202a-415f-a822-197b9f69aa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.264843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0b571-202a-415f-a822-197b9f69aa6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.275404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0b571-202a-415f-a822-197b9f69aa6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f2b0b571-202a-415f-a822-197b9f69aa6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:39.283210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:39.284099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:39.284863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:39.289222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 133/274] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:39.290918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0b571-202a-415f-a822-197b9f69aa6c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:39.317323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:39.373980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25344246-76fb-4475-a906-137683371233 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:39.396381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0b571-202a-415f-a822-197b9f69aa6c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:39.415071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6c9c5-be73-47cd-87f4-2050ebaa5719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:39.416251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6c9c5-be73-47cd-87f4-2050ebaa5719 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.416773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6c9c5-be73-47cd-87f4-2050ebaa5719 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.417378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6c9c5-be73-47cd-87f4-2050ebaa5719 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.418735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6c9c5-be73-47cd-87f4-2050ebaa5719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:39.421170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6c9c5-be73-47cd-87f4-2050ebaa5719 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:39.427738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 142/275] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:39.429087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6c9c5-be73-47cd-87f4-2050ebaa5719 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:39.432211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:39.432945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.433328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.433679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.434990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:39.434990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:39.436329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:39.449875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 134/276] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:39.451429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:39.461746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:39.537416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:39.543148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6c9c5-be73-47cd-87f4-2050ebaa5719 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:39.552705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a57b7c-a86b-4a44-9a2d-e638327d8761 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:39.578556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a3d1a-1588-422c-b66d-c8010dd0f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:39.579479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3d1a-1588-422c-b66d-c8010dd0f836 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.579764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3d1a-1588-422c-b66d-c8010dd0f836 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.580040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3d1a-1588-422c-b66d-c8010dd0f836 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.580736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a3d1a-1588-422c-b66d-c8010dd0f836 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:39.582213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a3d1a-1588-422c-b66d-c8010dd0f836 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:39.588147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc49aa7-15fa-45cd-8bf8-7593787ced37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:39.590031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49aa7-15fa-45cd-8bf8-7593787ced37 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.590954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49aa7-15fa-45cd-8bf8-7593787ced37 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.591449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc49aa7-15fa-45cd-8bf8-7593787ced37 None admin] PATH_INFO: `/v3/users/3a2992cd1b9f4b809ce877ad282d6f37` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.605969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc49aa7-15fa-45cd-8bf8-7593787ced37 None admin] RBAC: Authorizing `identity:get_user(user_id=3a2992cd1b9f4b809ce877ad282d6f37)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:39.607414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc49aa7-15fa-45cd-8bf8-7593787ced37 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:39.609679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 135/277] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:39 2022] GET /identity/v3/users/3a2992cd1b9f4b809ce877ad282d6f37 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:39.610957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586a4d-b237-478a-abc8-0091d1a75599 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.611245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586a4d-b237-478a-abc8-0091d1a75599 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.611520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1586a4d-b237-478a-abc8-0091d1a75599 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.613153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 136/278] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:39.615048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1586a4d-b237-478a-abc8-0091d1a75599 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:39.660623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 143/279] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:16:39 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:39.662311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22f80d-c8bf-4626-ba14-a6d54dfdc5e1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.662986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22f80d-c8bf-4626-ba14-a6d54dfdc5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.663360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22f80d-c8bf-4626-ba14-a6d54dfdc5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.682813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4a22f80d-c8bf-4626-ba14-a6d54dfdc5e1 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:39.684160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4a22f80d-c8bf-4626-ba14-a6d54dfdc5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:39.742580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1586a4d-b237-478a-abc8-0091d1a75599 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:39.769826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 144/280] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:39.786480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:39.787198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.787534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.787807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.788703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] RBAC: Adding query filter params (role.id=8e1fb3d93ff549a2899c16dd1130e587, user.id=3a2992cd1b9f4b809ce877ad282d6f37, scope.system=all) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:39.789018 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:39.790821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:39.791949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22f80d-c8bf-4626-ba14-a6d54dfdc5e1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:39.797954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 137/281] 10.222.0.41 () {56 vars in 1308 bytes} [Sat May 28 03:16:39 2022] GET /identity/v3/role_assignments?user.id=3a2992cd1b9f4b809ce877ad282d6f37&scope.system=all&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:39.809096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:39.916957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22f80d-c8bf-4626-ba14-a6d54dfdc5e1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:39.959428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32102e7-cbc0-4c5d-b5a8-380c5e4b6c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:39.960246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32102e7-cbc0-4c5d-b5a8-380c5e4b6c54 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:39.960536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32102e7-cbc0-4c5d-b5a8-380c5e4b6c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:39.960815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32102e7-cbc0-4c5d-b5a8-380c5e4b6c54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:39.976009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32102e7-cbc0-4c5d-b5a8-380c5e4b6c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a32102e7-cbc0-4c5d-b5a8-380c5e4b6c54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:39.984149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:39.985088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:39.986372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 145/282] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:16:39.991809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:40.005854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:40.106182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709d472-296b-4e44-bcb2-616739015b5e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:40.143863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:40.144559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:40.144840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:40.145130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:40.145893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:40.146203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:40.147585 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:40.159046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 138/283] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:40.165081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32102e7-cbc0-4c5d-b5a8-380c5e4b6c54 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:40.250490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:40.277764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32102e7-cbc0-4c5d-b5a8-380c5e4b6c54 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:40.316572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce4d08-6bf8-44eb-942d-042e3f71d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:40.317527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce4d08-6bf8-44eb-942d-042e3f71d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:40.317808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce4d08-6bf8-44eb-942d-042e3f71d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:40.318085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce4d08-6bf8-44eb-942d-042e3f71d9a9 None admin] PATH_INFO: `/v3/users/7ece33cfe43945929e9db5e5a9f6408a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:40.321811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce4d08-6bf8-44eb-942d-042e3f71d9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=7ece33cfe43945929e9db5e5a9f6408a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:40.323279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce4d08-6bf8-44eb-942d-042e3f71d9a9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:40.324913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 146/284] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:40 2022] GET /identity/v3/users/7ece33cfe43945929e9db5e5a9f6408a => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:40.331183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:40.404040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:40.436302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68b255-a05d-472c-a9fc-6906e721d190 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:40.476549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebddeb-c98f-431a-8b67-59840b9e226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:40.477448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebddeb-c98f-431a-8b67-59840b9e226b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:40.477728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebddeb-c98f-431a-8b67-59840b9e226b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:40.477996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebddeb-c98f-431a-8b67-59840b9e226b None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:40.479199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebddeb-c98f-431a-8b67-59840b9e226b None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:40.480669 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebddeb-c98f-431a-8b67-59840b9e226b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:40.482382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 139/285] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:40 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:40.488112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce4d08-6bf8-44eb-942d-042e3f71d9a9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:40.594086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce4d08-6bf8-44eb-942d-042e3f71d9a9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:40.646362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783286d7-06ff-4764-83a0-7a7994d60fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:40.647257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783286d7-06ff-4764-83a0-7a7994d60fb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:40.647547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783286d7-06ff-4764-83a0-7a7994d60fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:40.647822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783286d7-06ff-4764-83a0-7a7994d60fb3 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e/users/7ece33cfe43945929e9db5e5a9f6408a/roles/86bf554ccc4b46ee91870389659035e3` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:40.649830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783286d7-06ff-4764-83a0-7a7994d60fb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7998a56b1396402ab848761300b3785e, user_id=7ece33cfe43945929e9db5e5a9f6408a, role_id=86bf554ccc4b46ee91870389659035e3)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:40.651343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783286d7-06ff-4764-83a0-7a7994d60fb3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:40.763702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 147/286] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:16:40 2022] PUT /identity/v3/projects/7998a56b1396402ab848761300b3785e/users/7ece33cfe43945929e9db5e5a9f6408a/roles/86bf554ccc4b46ee91870389659035e3 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:41.386404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e706d2d-d226-452b-b02b-f514e43f700f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:41.387103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e706d2d-d226-452b-b02b-f514e43f700f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:41.387103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e706d2d-d226-452b-b02b-f514e43f700f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:41.388516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 140/287] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:41.394411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe2029-0121-491b-af85-b953d2eb9765 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:41.394891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe2029-0121-491b-af85-b953d2eb9765 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:41.395044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe2029-0121-491b-af85-b953d2eb9765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:41.400486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:41.422149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5bfe2029-0121-491b-af85-b953d2eb9765 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:41.423372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5bfe2029-0121-491b-af85-b953d2eb9765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:41.513760 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 148/288] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:41.544888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e706d2d-d226-452b-b02b-f514e43f700f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:41.677596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e706d2d-d226-452b-b02b-f514e43f700f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:41.726033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80280d69-416b-4cf1-81e8-9b995de19349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:41.727274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80280d69-416b-4cf1-81e8-9b995de19349 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:41.727784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80280d69-416b-4cf1-81e8-9b995de19349 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:41.728257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80280d69-416b-4cf1-81e8-9b995de19349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:41.738515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:41.743123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80280d69-416b-4cf1-81e8-9b995de19349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-80280d69-416b-4cf1-81e8-9b995de19349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:41.751486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:41.752349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:41.753419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 141/289] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:41.758493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe2029-0121-491b-af85-b953d2eb9765 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:41.907919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe2029-0121-491b-af85-b953d2eb9765 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:41.962223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:41.968316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:41.969004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:41.969326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:41.969598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:41.970356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:41.970665 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:41.972319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:41.985708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 149/290] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:41.990909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80280d69-416b-4cf1-81e8-9b995de19349 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:42.111895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80280d69-416b-4cf1-81e8-9b995de19349 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:42.147390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4efa67-79f4-4c8c-8d63-3f1b53413dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:42.148313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4efa67-79f4-4c8c-8d63-3f1b53413dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.148633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4efa67-79f4-4c8c-8d63-3f1b53413dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.148880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4efa67-79f4-4c8c-8d63-3f1b53413dd0 None admin] PATH_INFO: `/v3/users/3c27f9cba4324127a8697c6ee62a0320` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.150120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4efa67-79f4-4c8c-8d63-3f1b53413dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=3c27f9cba4324127a8697c6ee62a0320)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:42.151596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4efa67-79f4-4c8c-8d63-3f1b53413dd0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:42.153652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 142/291] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:41 2022] GET /identity/v3/users/3c27f9cba4324127a8697c6ee62a0320 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:42.161840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:42.328517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628f356-9563-4e28-9f0d-1ea4cbae2f6b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:42.368499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177837e2-53e5-4df5-8007-7cff5b51e5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:42.369442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177837e2-53e5-4df5-8007-7cff5b51e5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.369867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177837e2-53e5-4df5-8007-7cff5b51e5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.370287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177837e2-53e5-4df5-8007-7cff5b51e5b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.458153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:42.471242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177837e2-53e5-4df5-8007-7cff5b51e5b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:42.473216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177837e2-53e5-4df5-8007-7cff5b51e5b9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:42.485499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03d12-42d5-45a0-8b4f-6185343d0dd8 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.486333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03d12-42d5-45a0-8b4f-6185343d0dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.486776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e03d12-42d5-45a0-8b4f-6185343d0dd8 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.488502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 143/292] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:42.494320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c297b-7d8d-425c-abe4-89d91491779b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.494827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c297b-7d8d-425c-abe4-89d91491779b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.495249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c297b-7d8d-425c-abe4-89d91491779b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.537193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-272c297b-7d8d-425c-abe4-89d91491779b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:42.538415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-272c297b-7d8d-425c-abe4-89d91491779b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:42.649218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 150/293] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:42.650561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db1f9-f665-4aa3-8f66-d8beaa63c284 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.650858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db1f9-f665-4aa3-8f66-d8beaa63c284 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.651138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db1f9-f665-4aa3-8f66-d8beaa63c284 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.652456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 151/294] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:42.653331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db1f9-f665-4aa3-8f66-d8beaa63c284 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:42.654357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:42.659299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 144/295] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:42.660776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3f85-2f7f-4140-8c8c-8c0a206f4bb6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.661233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3f85-2f7f-4140-8c8c-8c0a206f4bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.661650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3f85-2f7f-4140-8c8c-8c0a206f4bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.690379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-924d3f85-2f7f-4140-8c8c-8c0a206f4bb6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:42.692661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-924d3f85-2f7f-4140-8c8c-8c0a206f4bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:42.767475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 145/296] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:42.768140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d3f85-2f7f-4140-8c8c-8c0a206f4bb6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:42.772759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:42.783712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db1f9-f665-4aa3-8f66-d8beaa63c284 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:42.821016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd250c-fd52-4bbf-bf81-a2e36c5d6bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:42.821899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd250c-fd52-4bbf-bf81-a2e36c5d6bcc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.822183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd250c-fd52-4bbf-bf81-a2e36c5d6bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.822459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd250c-fd52-4bbf-bf81-a2e36c5d6bcc None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.823633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd250c-fd52-4bbf-bf81-a2e36c5d6bcc None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:42.825083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd250c-fd52-4bbf-bf81-a2e36c5d6bcc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:42.829132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 152/297] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:42 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:42.829132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fa9fd-cefa-4518-b4d4-254eef3ff4e3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.829132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fa9fd-cefa-4518-b4d4-254eef3ff4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.829132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fa9fd-cefa-4518-b4d4-254eef3ff4e3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.833182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 153/298] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:42.833182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fa9fd-cefa-4518-b4d4-254eef3ff4e3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:42.878876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d3f85-2f7f-4140-8c8c-8c0a206f4bb6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:42.914376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:42.924195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a4829-feec-418f-8436-bffe8ce0b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:42.925012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a4829-feec-418f-8436-bffe8ce0b4ad None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.925319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a4829-feec-418f-8436-bffe8ce0b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.925595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a4829-feec-418f-8436-bffe8ce0b4ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.933743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fa9fd-cefa-4518-b4d4-254eef3ff4e3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:42.935742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a4829-feec-418f-8436-bffe8ce0b4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-859a4829-feec-418f-8436-bffe8ce0b4ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:42.943443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:42.944306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:42.945178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 146/299] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:42.945867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a4829-feec-418f-8436-bffe8ce0b4ad None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:42.976280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aa37b-3cfc-434c-a676-a0defdf1aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:42.977446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa37b-3cfc-434c-a676-a0defdf1aed0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.977926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa37b-3cfc-434c-a676-a0defdf1aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.978275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa37b-3cfc-434c-a676-a0defdf1aed0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:42.988663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aa37b-3cfc-434c-a676-a0defdf1aed0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-454aa37b-3cfc-434c-a676-a0defdf1aed0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:42.996387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:42.997287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:42.998092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 154/300] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:42.998787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008b47a-9fce-4e49-b47d-96a1bc4d422d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:42.999593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008b47a-9fce-4e49-b47d-96a1bc4d422d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:42.999593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008b47a-9fce-4e49-b47d-96a1bc4d422d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.016140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b008b47a-9fce-4e49-b47d-96a1bc4d422d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:43.017351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b008b47a-9fce-4e49-b47d-96a1bc4d422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:43.035071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:43.054752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a4829-feec-418f-8436-bffe8ce0b4ad None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.097128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 155/301] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:43.097128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008b47a-9fce-4e49-b47d-96a1bc4d422d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:43.097367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.097367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.097367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.097367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.097714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] RBAC: Adding query filter params (role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=7998a56b1396402ab848761300b3785e, user.id=3c27f9cba4324127a8697c6ee62a0320) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:43.097714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.097714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.114714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 147/302] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:42 2022] GET /identity/v3/role_assignments?user.id=3c27f9cba4324127a8697c6ee62a0320&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:43.115681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:43.178951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:43.223033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008b47a-9fce-4e49-b47d-96a1bc4d422d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.245602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980f07f-984b-4896-87e6-5046340da10d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.265790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.266799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.267231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.267649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.268569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:43.269028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.270793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.281163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 156/303] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:43.282353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:43.289525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.290215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.290500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.290788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.291548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:43.291860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.293282 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.306765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 148/304] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:43.306765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:43.307294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:43.414486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004717f6-f318-45cf-b371-ccf01320a010 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.416796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5dc2b-1ff9-4498-b6e4-78637e99fc7a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.436132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:43.453530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354701e4-9ddf-4744-890d-a4a7a558f19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.454571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354701e4-9ddf-4744-890d-a4a7a558f19b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.455138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354701e4-9ddf-4744-890d-a4a7a558f19b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.455688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354701e4-9ddf-4744-890d-a4a7a558f19b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.457192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354701e4-9ddf-4744-890d-a4a7a558f19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.457719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6866d-d23c-4b23-9624-59e13946f735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.458584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6866d-d23c-4b23-9624-59e13946f735 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.458861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6866d-d23c-4b23-9624-59e13946f735 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.459136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6866d-d23c-4b23-9624-59e13946f735 None admin] PATH_INFO: `/v3/users/3a2992cd1b9f4b809ce877ad282d6f37` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.459936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354701e4-9ddf-4744-890d-a4a7a558f19b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.460380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6866d-d23c-4b23-9624-59e13946f735 None admin] RBAC: Authorizing `identity:get_user(user_id=3a2992cd1b9f4b809ce877ad282d6f37)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.461796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6866d-d23c-4b23-9624-59e13946f735 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.462473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 157/305] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:43.463513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 149/306] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:43 2022] GET /identity/v3/users/3a2992cd1b9f4b809ce877ad282d6f37 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:43.463795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354701e4-9ddf-4744-890d-a4a7a558f19b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:43.468285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6866d-d23c-4b23-9624-59e13946f735 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:43.551378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:43.570103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354701e4-9ddf-4744-890d-a4a7a558f19b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.576569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6866d-d23c-4b23-9624-59e13946f735 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.605949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90751135-dda8-4e6c-9510-d67bb9277192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.606818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90751135-dda8-4e6c-9510-d67bb9277192 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.607099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90751135-dda8-4e6c-9510-d67bb9277192 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.607377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90751135-dda8-4e6c-9510-d67bb9277192 None admin] PATH_INFO: `/v3/users/83c5bb7c4440428f92c6035e8a51e58a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.611906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b81bb-3423-4a66-a01e-42e2db692e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.612568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b81bb-3423-4a66-a01e-42e2db692e3b None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.612853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b81bb-3423-4a66-a01e-42e2db692e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.613151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b81bb-3423-4a66-a01e-42e2db692e3b None admin] PATH_INFO: `/v3/groups` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.613842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b81bb-3423-4a66-a01e-42e2db692e3b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.615281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b81bb-3423-4a66-a01e-42e2db692e3b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.622691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90751135-dda8-4e6c-9510-d67bb9277192 None admin] RBAC: Authorizing `identity:get_user(user_id=83c5bb7c4440428f92c6035e8a51e58a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.624732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90751135-dda8-4e6c-9510-d67bb9277192 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.626532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 158/307] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:43 2022] GET /identity/v3/users/83c5bb7c4440428f92c6035e8a51e58a => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:43.627258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90751135-dda8-4e6c-9510-d67bb9277192 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:43.681722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 150/308] 10.222.0.41 () {60 vars in 1134 bytes} [Sat May 28 03:16:43 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:16:43.682542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b81bb-3423-4a66-a01e-42e2db692e3b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:43.732184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:43.743826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90751135-dda8-4e6c-9510-d67bb9277192 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.785812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63daf92-ec1f-454e-bb8a-e305c8973641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.786737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63daf92-ec1f-454e-bb8a-e305c8973641 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.787032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63daf92-ec1f-454e-bb8a-e305c8973641 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.787364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63daf92-ec1f-454e-bb8a-e305c8973641 None admin] PATH_INFO: `/v3/system/users/3a2992cd1b9f4b809ce877ad282d6f37/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.789794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63daf92-ec1f-454e-bb8a-e305c8973641 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3a2992cd1b9f4b809ce877ad282d6f37, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.791392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63daf92-ec1f-454e-bb8a-e305c8973641 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.797804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b81bb-3423-4a66-a01e-42e2db692e3b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:43.833983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c6455-c08e-4625-8e30-ae1c94ba934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:43.834643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6455-c08e-4625-8e30-ae1c94ba934f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.834919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6455-c08e-4625-8e30-ae1c94ba934f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.835188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6455-c08e-4625-8e30-ae1c94ba934f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.836028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c6455-c08e-4625-8e30-ae1c94ba934f None admin] RBAC: Adding query filter params (role.id=86bf554ccc4b46ee91870389659035e3, user.id=83c5bb7c4440428f92c6035e8a51e58a, scope.system=all) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:43.836334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c6455-c08e-4625-8e30-ae1c94ba934f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:43.837853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c6455-c08e-4625-8e30-ae1c94ba934f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:43.843487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 151/309] 10.222.0.41 () {56 vars in 1308 bytes} [Sat May 28 03:16:43 2022] GET /identity/v3/role_assignments?user.id=83c5bb7c4440428f92c6035e8a51e58a&scope.system=all&role.id=86bf554ccc4b46ee91870389659035e3 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:43.844809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce79fd9-ac63-4bd7-a6f2-15bdd6bf3aac None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.845113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce79fd9-ac63-4bd7-a6f2-15bdd6bf3aac None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.845396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce79fd9-ac63-4bd7-a6f2-15bdd6bf3aac None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.849167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 152/310] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:43.853167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9b659-2910-45ba-9846-8e80ac0edf06 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:43.853419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9b659-2910-45ba-9846-8e80ac0edf06 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:43.853780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9b659-2910-45ba-9846-8e80ac0edf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:43.873185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1cb9b659-2910-45ba-9846-8e80ac0edf06 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:43.879372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1cb9b659-2910-45ba-9846-8e80ac0edf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:43.900810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 159/311] 10.222.0.41 () {58 vars in 1334 bytes} [Sat May 28 03:16:43 2022] PUT /identity/v3/system/users/3a2992cd1b9f4b809ce877ad282d6f37/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:43.969723 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 153/312] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:43.992488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63daf92-ec1f-454e-bb8a-e305c8973641 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:44.005652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:44.113590 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63daf92-ec1f-454e-bb8a-e305c8973641 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:44.148837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7423fce-211d-44a0-9b25-3ccb64f659bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:44.149659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7423fce-211d-44a0-9b25-3ccb64f659bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:44.149943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7423fce-211d-44a0-9b25-3ccb64f659bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:44.150218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7423fce-211d-44a0-9b25-3ccb64f659bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:44.161223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7423fce-211d-44a0-9b25-3ccb64f659bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a7423fce-211d-44a0-9b25-3ccb64f659bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:44.168895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:44.169801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:44.170677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:44.172444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 160/313] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:44.176286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9b659-2910-45ba-9846-8e80ac0edf06 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:44.212563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:44.281492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9b659-2910-45ba-9846-8e80ac0edf06 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:44.325119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:44.325803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:44.326082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:44.326384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:44.327166 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:44.327473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:44.328866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:44.337944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 154/314] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:44.343161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7423fce-211d-44a0-9b25-3ccb64f659bb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:44.414326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:44.453423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7423fce-211d-44a0-9b25-3ccb64f659bb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:44.488515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd2e2-dfc6-4f99-8a86-9a8275cce12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:44.489418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd2e2-dfc6-4f99-8a86-9a8275cce12c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:44.489706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd2e2-dfc6-4f99-8a86-9a8275cce12c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:44.489984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd2e2-dfc6-4f99-8a86-9a8275cce12c None admin] PATH_INFO: `/v3/users/7ece33cfe43945929e9db5e5a9f6408a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:44.491187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd2e2-dfc6-4f99-8a86-9a8275cce12c None admin] RBAC: Authorizing `identity:get_user(user_id=7ece33cfe43945929e9db5e5a9f6408a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:44.492631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd2e2-dfc6-4f99-8a86-9a8275cce12c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:44.494538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 161/315] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:44 2022] GET /identity/v3/users/7ece33cfe43945929e9db5e5a9f6408a => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:44.500329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:44.594224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:44.600047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a3707-cbcf-47d0-8286-664670f15da5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:44.634710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c039b0d-92ac-465e-a86a-02cadd875202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:44.635361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c039b0d-92ac-465e-a86a-02cadd875202 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:44.635643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c039b0d-92ac-465e-a86a-02cadd875202 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:44.635918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c039b0d-92ac-465e-a86a-02cadd875202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:44.703618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c039b0d-92ac-465e-a86a-02cadd875202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:44.705180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c039b0d-92ac-465e-a86a-02cadd875202 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:44.796942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:44.807416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 155/316] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:44.808497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd2e2-dfc6-4f99-8a86-9a8275cce12c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:44.922330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd2e2-dfc6-4f99-8a86-9a8275cce12c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:44.951831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:44.956744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23775c-2747-49ee-8f9a-58e1cc3da754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:44.957692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23775c-2747-49ee-8f9a-58e1cc3da754 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:44.957983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23775c-2747-49ee-8f9a-58e1cc3da754 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:44.958323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23775c-2747-49ee-8f9a-58e1cc3da754 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:44.959571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23775c-2747-49ee-8f9a-58e1cc3da754 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:44.961048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23775c-2747-49ee-8f9a-58e1cc3da754 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:44.963189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 162/317] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:44 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:44.969559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c039b0d-92ac-465e-a86a-02cadd875202 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:45.072403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c039b0d-92ac-465e-a86a-02cadd875202 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:45.109127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2352e08-bacb-416c-839d-326bfa90a363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:45.110172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2352e08-bacb-416c-839d-326bfa90a363 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:45.110656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2352e08-bacb-416c-839d-326bfa90a363 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:45.111157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2352e08-bacb-416c-839d-326bfa90a363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:45.112207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2352e08-bacb-416c-839d-326bfa90a363 None admin] RBAC: Adding query filter params (role.id=86bf554ccc4b46ee91870389659035e3, scope.project.id=7998a56b1396402ab848761300b3785e, user.id=7ece33cfe43945929e9db5e5a9f6408a) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:45.112717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2352e08-bacb-416c-839d-326bfa90a363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:45.114395 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2352e08-bacb-416c-839d-326bfa90a363 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:45.121217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 156/318] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:16:44 2022] GET /identity/v3/role_assignments?user.id=7ece33cfe43945929e9db5e5a9f6408a&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=86bf554ccc4b46ee91870389659035e3 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:46.118970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b38f-3a51-4579-a6d1-7e424e7e7108 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.119476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b38f-3a51-4579-a6d1-7e424e7e7108 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.119549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b38f-3a51-4579-a6d1-7e424e7e7108 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.120909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 163/319] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:46.126508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed9fa6-14cb-4c25-9cb5-065b3d041244 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.126882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed9fa6-14cb-4c25-9cb5-065b3d041244 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.127073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed9fa6-14cb-4c25-9cb5-065b3d041244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.142223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40bd73-9ce3-4661-a021-b3e8754d8866 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.142514 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40bd73-9ce3-4661-a021-b3e8754d8866 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.142794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40bd73-9ce3-4661-a021-b3e8754d8866 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.143834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-21ed9fa6-14cb-4c25-9cb5-065b3d041244 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:46.144152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 164/320] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:46.145006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-21ed9fa6-14cb-4c25-9cb5-065b3d041244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:46.149052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f88c2-8274-4333-a2f7-a05e4fcad2c8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.149371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f88c2-8274-4333-a2f7-a05e4fcad2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.149654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f88c2-8274-4333-a2f7-a05e4fcad2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.166933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-789f88c2-8274-4333-a2f7-a05e4fcad2c8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:46.168037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-789f88c2-8274-4333-a2f7-a05e4fcad2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:46.203664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:46.215029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 157/321] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:46.216717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3140-6046-4d6e-8e90-a6ff625db675 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.217600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3140-6046-4d6e-8e90-a6ff625db675 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.217879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3140-6046-4d6e-8e90-a6ff625db675 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.219370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 158/322] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:46.225011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a26c-346e-4f88-a71c-b47d0833b779 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.225342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a26c-346e-4f88-a71c-b47d0833b779 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.225624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a26c-346e-4f88-a71c-b47d0833b779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.243038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-24e2a26c-346e-4f88-a71c-b47d0833b779 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:46.244420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-24e2a26c-346e-4f88-a71c-b47d0833b779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:46.248357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 165/323] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:46.249048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f88c2-8274-4333-a2f7-a05e4fcad2c8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:46.314448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 159/324] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:46.315280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a26c-346e-4f88-a71c-b47d0833b779 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:46.354389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f88c2-8274-4333-a2f7-a05e4fcad2c8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:46.383504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:46.388887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a969c77-2180-468c-9b60-7eabc5f93901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:46.389710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a969c77-2180-468c-9b60-7eabc5f93901 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.389994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a969c77-2180-468c-9b60-7eabc5f93901 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.390271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a969c77-2180-468c-9b60-7eabc5f93901 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.391448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a969c77-2180-468c-9b60-7eabc5f93901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:46.392883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a969c77-2180-468c-9b60-7eabc5f93901 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:46.394805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 166/325] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:16:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:46.395356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a969c77-2180-468c-9b60-7eabc5f93901 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:46.424553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a26c-346e-4f88-a71c-b47d0833b779 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:46.444012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:46.457210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b96ba-b884-4aa5-9d43-c85d81e239c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:46.457998 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b96ba-b884-4aa5-9d43-c85d81e239c3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.458371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b96ba-b884-4aa5-9d43-c85d81e239c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.458604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b96ba-b884-4aa5-9d43-c85d81e239c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.468963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b96ba-b884-4aa5-9d43-c85d81e239c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a50b96ba-b884-4aa5-9d43-c85d81e239c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:46.477699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:46.478577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:46.479364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 160/326] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:46.480010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b96ba-b884-4aa5-9d43-c85d81e239c3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:46.492735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a969c77-2180-468c-9b60-7eabc5f93901 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:46.525407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4be031-70fe-4525-b0b9-2c1aa93cda4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:46.526185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be031-70fe-4525-b0b9-2c1aa93cda4d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.526464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be031-70fe-4525-b0b9-2c1aa93cda4d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.526735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be031-70fe-4525-b0b9-2c1aa93cda4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.537157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4be031-70fe-4525-b0b9-2c1aa93cda4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2a4be031-70fe-4525-b0b9-2c1aa93cda4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:46.544836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:46.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:46.547439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 167/327] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:46.548218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4be031-70fe-4525-b0b9-2c1aa93cda4d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:46.585001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b96ba-b884-4aa5-9d43-c85d81e239c3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:46.604958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:46.617160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dd4ab-9ec7-4234-b427-8f560d662a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:46.617822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dd4ab-9ec7-4234-b427-8f560d662a17 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.618108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dd4ab-9ec7-4234-b427-8f560d662a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.618386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80dd4ab-9ec7-4234-b427-8f560d662a17 None admin] PATH_INFO: `/v3/groups` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.619058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80dd4ab-9ec7-4234-b427-8f560d662a17 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:46.620516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80dd4ab-9ec7-4234-b427-8f560d662a17 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:46.644904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4be031-70fe-4525-b0b9-2c1aa93cda4d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:46.677939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:46.678536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.678822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.679101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.679841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:46.680153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:46.681642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:46.689560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 168/328] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:46.690415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:46.737176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 161/329] 10.222.0.41 () {60 vars in 1134 bytes} [Sat May 28 03:16:46 2022] POST /identity/v3/groups => generated 226 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:16:46.737176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dd4ab-9ec7-4234-b427-8f560d662a17 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:46.775807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:46.812782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d341981-99d1-435f-bf60-9e89ce5dfbf0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:46.837941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:46.851552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80dd4ab-9ec7-4234-b427-8f560d662a17 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:46.853456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:46.854140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.854433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.854714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.855523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:46.855837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:46.857421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:46.867358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 169/330] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:46.873417 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:46.899100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe705f-b0bb-4830-aa45-1ad2b3add98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:46.903636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe705f-b0bb-4830-aa45-1ad2b3add98b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:46.903636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe705f-b0bb-4830-aa45-1ad2b3add98b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:46.903636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe705f-b0bb-4830-aa45-1ad2b3add98b None admin] PATH_INFO: `/v3/users/83c5bb7c4440428f92c6035e8a51e58a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:46.903636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe705f-b0bb-4830-aa45-1ad2b3add98b None admin] RBAC: Authorizing `identity:get_user(user_id=83c5bb7c4440428f92c6035e8a51e58a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:46.904303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe705f-b0bb-4830-aa45-1ad2b3add98b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:46.911197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 162/331] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:46 2022] GET /identity/v3/users/83c5bb7c4440428f92c6035e8a51e58a => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:46.912379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe705f-b0bb-4830-aa45-1ad2b3add98b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:47.005182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f58b55-c8a3-432f-bd4f-beb158cd86b3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:47.033195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe705f-b0bb-4830-aa45-1ad2b3add98b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:47.043521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:47.052274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf72c-d074-4188-bcbd-369f3e307c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:47.053184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf72c-d074-4188-bcbd-369f3e307c69 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:47.053477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf72c-d074-4188-bcbd-369f3e307c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:47.053759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf72c-d074-4188-bcbd-369f3e307c69 None admin] PATH_INFO: `/v3/users/3a2992cd1b9f4b809ce877ad282d6f37` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:47.057876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bf72c-d074-4188-bcbd-369f3e307c69 None admin] RBAC: Authorizing `identity:get_user(user_id=3a2992cd1b9f4b809ce877ad282d6f37)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:47.057876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bf72c-d074-4188-bcbd-369f3e307c69 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:47.059943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 170/332] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:46 2022] GET /identity/v3/users/3a2992cd1b9f4b809ce877ad282d6f37 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:47.066263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf72c-d074-4188-bcbd-369f3e307c69 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:47.075427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf376fa0-9902-4d49-8cc4-72d73f95f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:47.076904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf376fa0-9902-4d49-8cc4-72d73f95f1f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:47.077229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf376fa0-9902-4d49-8cc4-72d73f95f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:47.077509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf376fa0-9902-4d49-8cc4-72d73f95f1f3 None admin] PATH_INFO: `/v3/system/users/83c5bb7c4440428f92c6035e8a51e58a/roles/86bf554ccc4b46ee91870389659035e3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:47.080478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf376fa0-9902-4d49-8cc4-72d73f95f1f3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=83c5bb7c4440428f92c6035e8a51e58a, role_id=86bf554ccc4b46ee91870389659035e3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:47.082259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf376fa0-9902-4d49-8cc4-72d73f95f1f3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:47.126300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:47.146389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 163/333] 10.222.0.41 () {58 vars in 1334 bytes} [Sat May 28 03:16:46 2022] PUT /identity/v3/system/users/83c5bb7c4440428f92c6035e8a51e58a/roles/86bf554ccc4b46ee91870389659035e3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:16:47.185225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf72c-d074-4188-bcbd-369f3e307c69 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:47.224988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed3a89-7fa7-498c-9971-926774cd3f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:47.225690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed3a89-7fa7-498c-9971-926774cd3f94 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:47.225972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed3a89-7fa7-498c-9971-926774cd3f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:47.226247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed3a89-7fa7-498c-9971-926774cd3f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:47.227222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed3a89-7fa7-498c-9971-926774cd3f94 None admin] RBAC: Adding query filter params (role.id=8e1fb3d93ff549a2899c16dd1130e587, user.id=3a2992cd1b9f4b809ce877ad282d6f37, scope.system=all) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:47.227535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed3a89-7fa7-498c-9971-926774cd3f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:47.229006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfed3a89-7fa7-498c-9971-926774cd3f94 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:47.243352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 171/334] 10.222.0.41 () {56 vars in 1308 bytes} [Sat May 28 03:16:47 2022] GET /identity/v3/role_assignments?user.id=3a2992cd1b9f4b809ce877ad282d6f37&scope.system=all&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:49.621001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67073-c296-486c-8666-b8f1bdec416c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:49.621557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67073-c296-486c-8666-b8f1bdec416c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:49.621557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67073-c296-486c-8666-b8f1bdec416c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:49.623235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 164/335] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:49.630649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04cc5-9540-4403-98e9-9419efedb19c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:49.631084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04cc5-9540-4403-98e9-9419efedb19c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:49.631282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04cc5-9540-4403-98e9-9419efedb19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:49.655714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-86c04cc5-9540-4403-98e9-9419efedb19c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:49.656893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-86c04cc5-9540-4403-98e9-9419efedb19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:49.738260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 172/336] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:49.739509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e27f7b-639e-49b9-8e85-b5a88a9cf6f4 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:49.739804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e27f7b-639e-49b9-8e85-b5a88a9cf6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:49.740084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e27f7b-639e-49b9-8e85-b5a88a9cf6f4 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:49.746048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 165/337] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:49.754925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d8a23-a411-4fd3-8db1-cf9d1e0fd59b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:49.755536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d8a23-a411-4fd3-8db1-cf9d1e0fd59b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:49.756031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d8a23-a411-4fd3-8db1-cf9d1e0fd59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:49.777637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-951d8a23-a411-4fd3-8db1-cf9d1e0fd59b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:49.779168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-951d8a23-a411-4fd3-8db1-cf9d1e0fd59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:49.783939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e27f7b-639e-49b9-8e85-b5a88a9cf6f4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:49.818821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:49.878451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 173/338] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:49.919427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d8a23-a411-4fd3-8db1-cf9d1e0fd59b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:49.963055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e27f7b-639e-49b9-8e85-b5a88a9cf6f4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:50.017923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa88c2-f5fd-49e0-a9c8-62f529880186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:50.018727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa88c2-f5fd-49e0-a9c8-62f529880186 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.018996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa88c2-f5fd-49e0-a9c8-62f529880186 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.019269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa88c2-f5fd-49e0-a9c8-62f529880186 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.035312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa88c2-f5fd-49e0-a9c8-62f529880186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-97aa88c2-f5fd-49e0-a9c8-62f529880186 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:50.042244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:50.043108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:50.043866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 166/339] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:50.049729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa88c2-f5fd-49e0-a9c8-62f529880186 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:50.141983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:50.148879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951d8a23-a411-4fd3-8db1-cf9d1e0fd59b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:50.210535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb36bf-a19a-4e7b-acc5-9fd39695e55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:50.211732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb36bf-a19a-4e7b-acc5-9fd39695e55a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.212212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb36bf-a19a-4e7b-acc5-9fd39695e55a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.212744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb36bf-a19a-4e7b-acc5-9fd39695e55a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.235105 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb36bf-a19a-4e7b-acc5-9fd39695e55a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bfcb36bf-a19a-4e7b-acc5-9fd39695e55a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:50.248848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:50.249774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:50.250523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:50.252874 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 174/340] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:50.261275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb36bf-a19a-4e7b-acc5-9fd39695e55a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:50.305972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa88c2-f5fd-49e0-a9c8-62f529880186 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:50.317481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:50.360546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cdd23-d6b2-41a3-8e3c-de100640eda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:50.361437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cdd23-d6b2-41a3-8e3c-de100640eda2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.361821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cdd23-d6b2-41a3-8e3c-de100640eda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.362200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cdd23-d6b2-41a3-8e3c-de100640eda2 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.363028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cdd23-d6b2-41a3-8e3c-de100640eda2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:50.363434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cdd23-d6b2-41a3-8e3c-de100640eda2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:50.365546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cdd23-d6b2-41a3-8e3c-de100640eda2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:50.378167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 167/341] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:50.379717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f222-f0f6-413f-a4df-abb0344d1087 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.380126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f222-f0f6-413f-a4df-abb0344d1087 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.380500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72f222-f0f6-413f-a4df-abb0344d1087 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.382250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 168/342] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:50.389196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72f222-f0f6-413f-a4df-abb0344d1087 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:50.408561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb36bf-a19a-4e7b-acc5-9fd39695e55a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:50.447264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37a5ec-4519-4c5a-93d3-0c8d82c29e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:50.449361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37a5ec-4519-4c5a-93d3-0c8d82c29e80 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.450004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37a5ec-4519-4c5a-93d3-0c8d82c29e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.451038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37a5ec-4519-4c5a-93d3-0c8d82c29e80 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.452821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37a5ec-4519-4c5a-93d3-0c8d82c29e80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:50.456688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37a5ec-4519-4c5a-93d3-0c8d82c29e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:50.458250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37a5ec-4519-4c5a-93d3-0c8d82c29e80 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:50.472784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 175/343] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:50.475634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bdcd2-bb29-43d1-9699-c63113249c44 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.476873 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bdcd2-bb29-43d1-9699-c63113249c44 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.478073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bdcd2-bb29-43d1-9699-c63113249c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.500872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a91bdcd2-bb29-43d1-9699-c63113249c44 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:50.502820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a91bdcd2-bb29-43d1-9699-c63113249c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:50.534866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:50.542471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72f222-f0f6-413f-a4df-abb0344d1087 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:50.588465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cb8df-b941-4a73-9cc0-581d5221544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:50.589260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb8df-b941-4a73-9cc0-581d5221544e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.589560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb8df-b941-4a73-9cc0-581d5221544e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.589910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cb8df-b941-4a73-9cc0-581d5221544e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.591630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 176/344] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:50.594051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c076-d55e-4bc2-9268-2ca4e3944701 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.595451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c076-d55e-4bc2-9268-2ca4e3944701 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.595883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c076-d55e-4bc2-9268-2ca4e3944701 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.597769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 177/345] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:50.598609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74c076-d55e-4bc2-9268-2ca4e3944701 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:50.625464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:50.683961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cb8df-b941-4a73-9cc0-581d5221544e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:50.686212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cb8df-b941-4a73-9cc0-581d5221544e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:50.729789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74c076-d55e-4bc2-9268-2ca4e3944701 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:50.772970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cf077-92de-4d0c-8c85-e53c9576433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:50.773908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cf077-92de-4d0c-8c85-e53c9576433a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.774331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cf077-92de-4d0c-8c85-e53c9576433a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.774747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cf077-92de-4d0c-8c85-e53c9576433a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.814310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:50.823754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 169/346] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 03:16:50.829601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb35dda-0f10-4413-ab9c-ed76e822adc4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:50.830889 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb35dda-0f10-4413-ab9c-ed76e822adc4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:50.831511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb35dda-0f10-4413-ab9c-ed76e822adc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:50.852044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-efb35dda-0f10-4413-ab9c-ed76e822adc4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:50.853589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-efb35dda-0f10-4413-ab9c-ed76e822adc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:50.866495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cf077-92de-4d0c-8c85-e53c9576433a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:50.868275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cf077-92de-4d0c-8c85-e53c9576433a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:50.894503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:50.941808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 170/347] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:50.942454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb35dda-0f10-4413-ab9c-ed76e822adc4 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:50.987300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 178/348] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:50.988156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cf077-92de-4d0c-8c85-e53c9576433a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.077533 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb35dda-0f10-4413-ab9c-ed76e822adc4 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.094715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cf077-92de-4d0c-8c85-e53c9576433a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.102944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:51.114407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524e83-670b-42ac-b94d-386dac106fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.115256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524e83-670b-42ac-b94d-386dac106fab None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.115548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524e83-670b-42ac-b94d-386dac106fab None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.115831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524e83-670b-42ac-b94d-386dac106fab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.116237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:51.126679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524e83-670b-42ac-b94d-386dac106fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:51.130893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c54e9b-4fe7-4125-8f37-c0730a0f9a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.131749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c54e9b-4fe7-4125-8f37-c0730a0f9a61 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.132026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c54e9b-4fe7-4125-8f37-c0730a0f9a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.132296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c54e9b-4fe7-4125-8f37-c0730a0f9a61 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.133511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c54e9b-4fe7-4125-8f37-c0730a0f9a61 None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4f524e83-670b-42ac-b94d-386dac106fab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:51.134725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:51.135594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:51.136384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:51.137071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c54e9b-4fe7-4125-8f37-c0730a0f9a61 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:51.137071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 179/349] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:50 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:51.137813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c54e9b-4fe7-4125-8f37-c0730a0f9a61 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.143421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 171/350] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:51.143421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524e83-670b-42ac-b94d-386dac106fab None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.277393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524e83-670b-42ac-b94d-386dac106fab None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.277896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c54e9b-4fe7-4125-8f37-c0730a0f9a61 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.317660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b3eee-c4d6-45f9-bb5c-e1e9b7f08278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.318472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3eee-c4d6-45f9-bb5c-e1e9b7f08278 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.318754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3eee-c4d6-45f9-bb5c-e1e9b7f08278 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.319027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b3eee-c4d6-45f9-bb5c-e1e9b7f08278 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.319473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db9b9c-20ac-4544-84fe-586349f62e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.320199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db9b9c-20ac-4544-84fe-586349f62e7e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.320478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db9b9c-20ac-4544-84fe-586349f62e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.320748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db9b9c-20ac-4544-84fe-586349f62e7e None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.323015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db9b9c-20ac-4544-84fe-586349f62e7e None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:51.324481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db9b9c-20ac-4544-84fe-586349f62e7e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:51.326870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 180/351] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:51.328844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31b89-17f7-4d0a-9dc2-04cc3b80250d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.328844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31b89-17f7-4d0a-9dc2-04cc3b80250d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.329025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b31b89-17f7-4d0a-9dc2-04cc3b80250d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.334603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 181/352] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:51.335339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b31b89-17f7-4d0a-9dc2-04cc3b80250d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.338605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b3eee-c4d6-45f9-bb5c-e1e9b7f08278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-512b3eee-c4d6-45f9-bb5c-e1e9b7f08278 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:51.350397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:51.351404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:51.352170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 172/353] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:51.353563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b3eee-c4d6-45f9-bb5c-e1e9b7f08278 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.413327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:51.417223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:51.471646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b31b89-17f7-4d0a-9dc2-04cc3b80250d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.492978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b3eee-c4d6-45f9-bb5c-e1e9b7f08278 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.511555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c66e2-424d-4737-a887-f840676ca241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.512443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c66e2-424d-4737-a887-f840676ca241 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.512727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c66e2-424d-4737-a887-f840676ca241 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.512997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c66e2-424d-4737-a887-f840676ca241 None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.519606 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c66e2-424d-4737-a887-f840676ca241 None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:51.521288 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c66e2-424d-4737-a887-f840676ca241 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:51.523558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 182/354] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:51.523709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c66e2-424d-4737-a887-f840676ca241 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.538198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0332a9-d641-4726-9895-c5481596c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.538803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0332a9-d641-4726-9895-c5481596c0cd None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.539082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0332a9-d641-4726-9895-c5481596c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.539361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0332a9-d641-4726-9895-c5481596c0cd None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.540236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0332a9-d641-4726-9895-c5481596c0cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:51.540550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0332a9-d641-4726-9895-c5481596c0cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:51.542275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0332a9-d641-4726-9895-c5481596c0cd None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:51.551943 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 173/355] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:51.553616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2825eb1-0d04-4f9e-8264-eedd325181fc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.553909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2825eb1-0d04-4f9e-8264-eedd325181fc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.554176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2825eb1-0d04-4f9e-8264-eedd325181fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.571326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f2825eb1-0d04-4f9e-8264-eedd325181fc None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:51.572505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f2825eb1-0d04-4f9e-8264-eedd325181fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:51.646214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c66e2-424d-4737-a887-f840676ca241 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.653378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 174/356] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:51.655051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2825eb1-0d04-4f9e-8264-eedd325181fc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.670904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:51.689872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0623841-29e6-493d-b8bf-5594d396cd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.690728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0623841-29e6-493d-b8bf-5594d396cd05 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.691010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0623841-29e6-493d-b8bf-5594d396cd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.691417 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0623841-29e6-493d-b8bf-5594d396cd05 None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.692691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0623841-29e6-493d-b8bf-5594d396cd05 None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:51.694561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0623841-29e6-493d-b8bf-5594d396cd05 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:51.695432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:51.698059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 183/357] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:51.699507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d20a7-1d8c-47fd-9e85-98d26b0317c8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.699932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d20a7-1d8c-47fd-9e85-98d26b0317c8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.700828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d20a7-1d8c-47fd-9e85-98d26b0317c8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.702967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 184/358] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:51.703861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d20a7-1d8c-47fd-9e85-98d26b0317c8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.771230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2825eb1-0d04-4f9e-8264-eedd325181fc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.814294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.814885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.815524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.815524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.816161 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:51.816462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:51.817958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:51.826990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 175/359] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:51.827833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.832027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d20a7-1d8c-47fd-9e85-98d26b0317c8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.878707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:51.879357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:51.879635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:51.879987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:51.880734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] RBAC: Adding query filter params (group.id=394f7a2fceff4e9cb7c17083c18e2f5f, role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=cd271be4aaeb4efab5ef86157a14122b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:51.881040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:51.882539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:51.888181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 185/360] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/role_assignments?group.id=394f7a2fceff4e9cb7c17083c18e2f5f&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:51.889002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:51.948483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:51.956406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f94f-ca85-4baa-9f5b-7ab73ee9959e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:51.961603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:52.002345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44736930-f07d-49e3-9f99-a37db37721ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.003004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44736930-f07d-49e3-9f99-a37db37721ed None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.003283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44736930-f07d-49e3-9f99-a37db37721ed None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.003556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44736930-f07d-49e3-9f99-a37db37721ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.025215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf17d-a590-450b-8c7a-2adc824da30d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:52.073706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf18edc-d85a-4e46-9b9c-f8a801fbdf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.074447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf18edc-d85a-4e46-9b9c-f8a801fbdf54 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.074727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf18edc-d85a-4e46-9b9c-f8a801fbdf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.074999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf18edc-d85a-4e46-9b9c-f8a801fbdf54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.091175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf18edc-d85a-4e46-9b9c-f8a801fbdf54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:52.097477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44736930-f07d-49e3-9f99-a37db37721ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:52.098933 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44736930-f07d-49e3-9f99-a37db37721ed None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5bf18edc-d85a-4e46-9b9c-f8a801fbdf54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:52.100396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:52.101317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:52.102292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:52.108254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 186/361] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:52.108254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf18edc-d85a-4e46-9b9c-f8a801fbdf54 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:52.236099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:52.244601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 176/362] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:52.250384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3001af-b562-4740-9261-b94dc7ec280a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.250384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3001af-b562-4740-9261-b94dc7ec280a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.250384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3001af-b562-4740-9261-b94dc7ec280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.250683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf18edc-d85a-4e46-9b9c-f8a801fbdf54 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:52.265496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:52.266847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bb3001af-b562-4740-9261-b94dc7ec280a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:52.268372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bb3001af-b562-4740-9261-b94dc7ec280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:52.297301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.298126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.298767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.299052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.300353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] RBAC: Adding query filter params (group.id=394f7a2fceff4e9cb7c17083c18e2f5f, role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=cd271be4aaeb4efab5ef86157a14122b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:52.300785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:52.302885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:52.310799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 187/363] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:16:52 2022] GET /identity/v3/role_assignments?group.id=394f7a2fceff4e9cb7c17083c18e2f5f&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:52.311841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:52.364683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 177/364] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:52.366930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3001af-b562-4740-9261-b94dc7ec280a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:52.458661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e26fe-b7b8-46e9-a5c5-844db31b93f6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:52.502280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3001af-b562-4740-9261-b94dc7ec280a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:52.515479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58d6b3-0d55-430b-b577-aa972afd4ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.516499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58d6b3-0d55-430b-b577-aa972afd4ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.516855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58d6b3-0d55-430b-b577-aa972afd4ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.517246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58d6b3-0d55-430b-b577-aa972afd4ed7 None admin] PATH_INFO: `/v3/users/83c5bb7c4440428f92c6035e8a51e58a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.519307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58d6b3-0d55-430b-b577-aa972afd4ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=83c5bb7c4440428f92c6035e8a51e58a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:52.520875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58d6b3-0d55-430b-b577-aa972afd4ed7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:52.525871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 188/365] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:16:52 2022] GET /identity/v3/users/83c5bb7c4440428f92c6035e8a51e58a => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:52.526980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58d6b3-0d55-430b-b577-aa972afd4ed7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:52.556391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:52.565880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.567211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.567474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.567750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.568501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:52.568812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:52.570232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:52.580951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 178/366] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:52.582900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:52.596567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:52.677960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58d6b3-0d55-430b-b577-aa972afd4ed7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:52.716297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd817d7-340f-4ebf-9433-848aa3001f8d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:52.716903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eed0b-bf90-4964-a901-98ed74bb4567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.719167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eed0b-bf90-4964-a901-98ed74bb4567 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.719563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eed0b-bf90-4964-a901-98ed74bb4567 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.720525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eed0b-bf90-4964-a901-98ed74bb4567 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.721338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792eed0b-bf90-4964-a901-98ed74bb4567 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:52.723255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792eed0b-bf90-4964-a901-98ed74bb4567 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:52.728880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 189/367] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:52 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:52.728880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eed0b-bf90-4964-a901-98ed74bb4567 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:52.769328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedc74a-8d88-4a6c-950f-f9da77c2fdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.771226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedc74a-8d88-4a6c-950f-f9da77c2fdef None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.771694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedc74a-8d88-4a6c-950f-f9da77c2fdef None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.772151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedc74a-8d88-4a6c-950f-f9da77c2fdef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.786465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedc74a-8d88-4a6c-950f-f9da77c2fdef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9dedc74a-8d88-4a6c-950f-f9da77c2fdef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:52.797364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:52.798551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:52.799569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:52.802537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 179/368] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:52.803599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedc74a-8d88-4a6c-950f-f9da77c2fdef None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:52.865059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eed0b-bf90-4964-a901-98ed74bb4567 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:52.875272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:52.885390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:52.945810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.945810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.945810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.945810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:52.946219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] RBAC: Adding query filter params (role.id=86bf554ccc4b46ee91870389659035e3, user.id=83c5bb7c4440428f92c6035e8a51e58a, scope.system=all) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:52.946219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:52.946219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:52.946419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedc74a-8d88-4a6c-950f-f9da77c2fdef None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:52.948548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 190/369] 10.222.0.41 () {56 vars in 1308 bytes} [Sat May 28 03:16:52 2022] GET /identity/v3/role_assignments?user.id=83c5bb7c4440428f92c6035e8a51e58a&scope.system=all&role.id=86bf554ccc4b46ee91870389659035e3 => generated 510 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:52.949648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:52.985518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315aa18d-0ecd-414a-b045-c4aff58859f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:52.986486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315aa18d-0ecd-414a-b045-c4aff58859f4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:52.986770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315aa18d-0ecd-414a-b045-c4aff58859f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:52.987048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315aa18d-0ecd-414a-b045-c4aff58859f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:53.075333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ab78e-66c2-4f73-bebc-e4ae09dfea98 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:53.076698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315aa18d-0ecd-414a-b045-c4aff58859f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:53.079263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315aa18d-0ecd-414a-b045-c4aff58859f4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:53.118816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a10382-c3d7-479b-aaf1-68e569add174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:53.120074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a10382-c3d7-479b-aaf1-68e569add174 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:53.120583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a10382-c3d7-479b-aaf1-68e569add174 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:53.121084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a10382-c3d7-479b-aaf1-68e569add174 None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:53.122646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a10382-c3d7-479b-aaf1-68e569add174 None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:53.124544 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a10382-c3d7-479b-aaf1-68e569add174 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:53.128079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 191/370] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:52 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:53.129322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a10382-c3d7-479b-aaf1-68e569add174 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:53.143556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:53.202717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:53.205997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 180/371] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:53.207061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315aa18d-0ecd-414a-b045-c4aff58859f4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:53.317765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a10382-c3d7-479b-aaf1-68e569add174 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:53.368916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:53.369642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:53.369923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:53.370196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:53.370952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:53.371258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:53.372623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:53.379662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315aa18d-0ecd-414a-b045-c4aff58859f4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:53.394867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 192/372] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:53.396776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:53.438585 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:53.439243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:53.439766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:53.440057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:53.440884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] RBAC: Adding query filter params (group.id=394f7a2fceff4e9cb7c17083c18e2f5f, role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=7998a56b1396402ab848761300b3785e) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:53.441279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:53.443812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:53.453190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 181/373] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:16:53 2022] GET /identity/v3/role_assignments?group.id=394f7a2fceff4e9cb7c17083c18e2f5f&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 260 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:53.453190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:53.475862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:53.553027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:53.556472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b6a1a-9b79-4712-b7d1-967fe22d700f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:53.604505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a310059-8689-40f4-b490-094836898175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:53.605506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310059-8689-40f4-b490-094836898175 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:53.605784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310059-8689-40f4-b490-094836898175 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:53.606065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a310059-8689-40f4-b490-094836898175 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:53.607316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a310059-8689-40f4-b490-094836898175 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:53.607984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd44066-94c9-4e46-bed8-f780df2053f4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:53.608755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a310059-8689-40f4-b490-094836898175 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:53.615108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 193/374] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:53 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:53.617656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a310059-8689-40f4-b490-094836898175 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:53.653385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029c67-12dc-4d23-bafa-88d52923ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:53.655175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029c67-12dc-4d23-bafa-88d52923ba67 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:53.655902 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029c67-12dc-4d23-bafa-88d52923ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:53.656358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c029c67-12dc-4d23-bafa-88d52923ba67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:53.747412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c029c67-12dc-4d23-bafa-88d52923ba67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:53.749050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a310059-8689-40f4-b490-094836898175 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:53.750365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c029c67-12dc-4d23-bafa-88d52923ba67 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:53.774380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:53.796260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265fc3a-881c-402a-a6a3-50d92777722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:53.807330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265fc3a-881c-402a-a6a3-50d92777722a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:53.807330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265fc3a-881c-402a-a6a3-50d92777722a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:53.807330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265fc3a-881c-402a-a6a3-50d92777722a None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:53.807330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265fc3a-881c-402a-a6a3-50d92777722a None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:53.807330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265fc3a-881c-402a-a6a3-50d92777722a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:53.807330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 194/375] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:53 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:53.809455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265fc3a-881c-402a-a6a3-50d92777722a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:53.851449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:53.891299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 182/376] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:53.897734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029c67-12dc-4d23-bafa-88d52923ba67 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:53.953232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265fc3a-881c-402a-a6a3-50d92777722a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:53.999564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:54.000608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:54.001095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:54.001608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:54.002751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] RBAC: Adding query filter params (group.id=394f7a2fceff4e9cb7c17083c18e2f5f, role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=7998a56b1396402ab848761300b3785e) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:54.003334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:54.005084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:54.012031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 195/377] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:16:53 2022] GET /identity/v3/role_assignments?group.id=394f7a2fceff4e9cb7c17083c18e2f5f&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:54.070304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c029c67-12dc-4d23-bafa-88d52923ba67 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:54.085820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:54.109666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44841993-3f19-4819-9688-f1c839c6b2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:54.110545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44841993-3f19-4819-9688-f1c839c6b2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:54.110825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44841993-3f19-4819-9688-f1c839c6b2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:54.111098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44841993-3f19-4819-9688-f1c839c6b2a8 None admin] PATH_INFO: `/v3/projects/cc9d398b29a149a4ab99273f43b67fb2` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:54.112339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44841993-3f19-4819-9688-f1c839c6b2a8 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9d398b29a149a4ab99273f43b67fb2)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:54.113823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44841993-3f19-4819-9688-f1c839c6b2a8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:54.116078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 183/378] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:53 2022] GET /identity/v3/projects/cc9d398b29a149a4ab99273f43b67fb2 => generated 344 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:54.121613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:54.273605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:54.277088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcb84e-3e4f-4b52-a02a-8e1b45f3f1b2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:54.322465 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b380309-2364-4533-afa9-ddd634468ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:54.323611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b380309-2364-4533-afa9-ddd634468ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:54.324050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b380309-2364-4533-afa9-ddd634468ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:54.324471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b380309-2364-4533-afa9-ddd634468ab7 None admin] PATH_INFO: `/v3/groups/bf393a79636f4a40a9039e120d3d0162` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:54.334617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b380309-2364-4533-afa9-ddd634468ab7 None admin] RBAC: Authorizing `identity:get_group(group_id=bf393a79636f4a40a9039e120d3d0162)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:54.336119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b380309-2364-4533-afa9-ddd634468ab7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:54.338358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 196/379] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:54 2022] GET /identity/v3/groups/bf393a79636f4a40a9039e120d3d0162 => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:54.344001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44841993-3f19-4819-9688-f1c839c6b2a8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:54.423161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481593a4-6b1f-4e64-aed2-634b9b66c2c3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:54.423655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481593a4-6b1f-4e64-aed2-634b9b66c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:54.423957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481593a4-6b1f-4e64-aed2-634b9b66c2c3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:54.429807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 197/380] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:54.432710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398f7d-c874-40fa-87d9-9227d07cb0a4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:54.433005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398f7d-c874-40fa-87d9-9227d07cb0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:54.433313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b398f7d-c874-40fa-87d9-9227d07cb0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:54.458931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7b398f7d-c874-40fa-87d9-9227d07cb0a4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:54.460140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7b398f7d-c874-40fa-87d9-9227d07cb0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:54.477999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44841993-3f19-4819-9688-f1c839c6b2a8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:54.519928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:54.520919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:54.521351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:54.521724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:54.523138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] RBAC: Adding query filter params (group.id=bf393a79636f4a40a9039e120d3d0162, role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=cc9d398b29a149a4ab99273f43b67fb2) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:54.523558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:54.527896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:54.536425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 184/381] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:16:54 2022] GET /identity/v3/role_assignments?group.id=bf393a79636f4a40a9039e120d3d0162&scope.project.id=cc9d398b29a149a4ab99273f43b67fb2&role.id=bb0756f8af694633b9743a958eb46e7a => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:54.564008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 198/382] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:54.602652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:54.670519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:54.746478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd9867-632e-43e3-b4c8-5cc6ab66c647 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:54.782747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aacaa-2e3d-4e69-9c3f-2cffdec1830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:54.784225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aacaa-2e3d-4e69-9c3f-2cffdec1830c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:54.784660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aacaa-2e3d-4e69-9c3f-2cffdec1830c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:54.785084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aacaa-2e3d-4e69-9c3f-2cffdec1830c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:54.797320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aacaa-2e3d-4e69-9c3f-2cffdec1830c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-5b3aacaa-2e3d-4e69-9c3f-2cffdec1830c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:54.805523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:54.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:54.807650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:54.810045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 185/383] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:54.819013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b398f7d-c874-40fa-87d9-9227d07cb0a4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:54.927559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:54.938943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b398f7d-c874-40fa-87d9-9227d07cb0a4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:54.980506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:54.981186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:54.981481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:54.981762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:54.982540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:54.982848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:54.984205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:54.992809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 199/384] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:54.998456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aacaa-2e3d-4e69-9c3f-2cffdec1830c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:55.112509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aacaa-2e3d-4e69-9c3f-2cffdec1830c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:55.120905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:55.160706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f285e6-33ff-49ff-9765-c78b699f73e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:55.161666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f285e6-33ff-49ff-9765-c78b699f73e1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:55.162421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f285e6-33ff-49ff-9765-c78b699f73e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:55.162708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f285e6-33ff-49ff-9765-c78b699f73e1 None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:55.164011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f285e6-33ff-49ff-9765-c78b699f73e1 None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:55.166057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f285e6-33ff-49ff-9765-c78b699f73e1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:55.167987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 186/385] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:54 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:55.173530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:55.296228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbc902-03a6-48ba-a4d8-f8b8e8ae3fcd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:55.339265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7cbf0-6671-48de-8e41-2142f63119e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:55.340530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7cbf0-6671-48de-8e41-2142f63119e3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:55.341025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7cbf0-6671-48de-8e41-2142f63119e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:55.342116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7cbf0-6671-48de-8e41-2142f63119e3 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:55.343569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7cbf0-6671-48de-8e41-2142f63119e3 None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:55.345234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7cbf0-6671-48de-8e41-2142f63119e3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:55.352171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 200/386] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:55 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:55.353708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f285e6-33ff-49ff-9765-c78b699f73e1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:55.370352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95a69b-6b49-4d40-aba3-79971b8e01b7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:55.370954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95a69b-6b49-4d40-aba3-79971b8e01b7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:55.371445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95a69b-6b49-4d40-aba3-79971b8e01b7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:55.373363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 201/387] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:55.377942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89277a5-a77b-433c-a95b-58c248c2729e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:55.378474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89277a5-a77b-433c-a95b-58c248c2729e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:55.378948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89277a5-a77b-433c-a95b-58c248c2729e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:55.401936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e89277a5-a77b-433c-a95b-58c248c2729e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:55.406001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e89277a5-a77b-433c-a95b-58c248c2729e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:55.440233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:55.535115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 202/388] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:55.538468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f285e6-33ff-49ff-9765-c78b699f73e1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:55.555421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89277a5-a77b-433c-a95b-58c248c2729e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:55.614830 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91f347-031b-4c46-9cdd-d8e7595827bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:55.616149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91f347-031b-4c46-9cdd-d8e7595827bb None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:55.616662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91f347-031b-4c46-9cdd-d8e7595827bb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:55.617158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91f347-031b-4c46-9cdd-d8e7595827bb None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b/groups/394f7a2fceff4e9cb7c17083c18e2f5f/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:55.622164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91f347-031b-4c46-9cdd-d8e7595827bb None admin] RBAC: Authorizing `identity:create_grant(project_id=cd271be4aaeb4efab5ef86157a14122b, group_id=394f7a2fceff4e9cb7c17083c18e2f5f, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:55.623989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91f347-031b-4c46-9cdd-d8e7595827bb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:55.689277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89277a5-a77b-433c-a95b-58c248c2729e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:55.706358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 187/389] 10.222.0.41 () {58 vars in 1442 bytes} [Sat May 28 03:16:55 2022] PUT /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b/groups/394f7a2fceff4e9cb7c17083c18e2f5f/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:55.737092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4f85b-3d6f-40dd-a5ef-ec3cdfdce8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:55.737956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f85b-3d6f-40dd-a5ef-ec3cdfdce8cc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:55.738240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f85b-3d6f-40dd-a5ef-ec3cdfdce8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:55.738519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4f85b-3d6f-40dd-a5ef-ec3cdfdce8cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:55.754552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4f85b-3d6f-40dd-a5ef-ec3cdfdce8cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4ac4f85b-3d6f-40dd-a5ef-ec3cdfdce8cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:55.767720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:55.768705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:55.769950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 203/390] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:16:55.775252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91f347-031b-4c46-9cdd-d8e7595827bb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:55.840183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:55.943972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91f347-031b-4c46-9cdd-d8e7595827bb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:55.990710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:55.991781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:55.992267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:55.992742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:55.993751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:55.994264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:55.995975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:56.006214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 188/391] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:56.011394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4f85b-3d6f-40dd-a5ef-ec3cdfdce8cc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:56.035804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:56.133995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4f85b-3d6f-40dd-a5ef-ec3cdfdce8cc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:56.176574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a54a18-8fc3-417d-8d89-c42dc068c3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:56.177357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a54a18-8fc3-417d-8d89-c42dc068c3bf None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:56.177643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a54a18-8fc3-417d-8d89-c42dc068c3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:56.177918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a54a18-8fc3-417d-8d89-c42dc068c3bf None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:56.179088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a54a18-8fc3-417d-8d89-c42dc068c3bf None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:56.180541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a54a18-8fc3-417d-8d89-c42dc068c3bf None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:56.182178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 204/392] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:56 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:56.188208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:56.336921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:56.344591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b9c5b-4b82-4ae3-9415-899e8f3463ad None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:56.394611 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d4771-7204-4193-bcfa-76257dd7956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:56.395856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d4771-7204-4193-bcfa-76257dd7956e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:56.396361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d4771-7204-4193-bcfa-76257dd7956e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:56.396845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d4771-7204-4193-bcfa-76257dd7956e None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:56.401818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d4771-7204-4193-bcfa-76257dd7956e None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:56.403510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d4771-7204-4193-bcfa-76257dd7956e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:56.405481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 189/393] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:56 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:56.411090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a54a18-8fc3-417d-8d89-c42dc068c3bf None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:56.549385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:56.578570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a54a18-8fc3-417d-8d89-c42dc068c3bf None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:56.629969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88c6f6-428c-45b8-84f7-98c79d80cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:56.630851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88c6f6-428c-45b8-84f7-98c79d80cb9f None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:56.631134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88c6f6-428c-45b8-84f7-98c79d80cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:56.631410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88c6f6-428c-45b8-84f7-98c79d80cb9f None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b/groups/394f7a2fceff4e9cb7c17083c18e2f5f/roles/fc5d6a08fc2048f19c14f691e35cd41c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:56.637229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8f04c-df45-4ca0-a2fc-34d95ecc039f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:56.637397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88c6f6-428c-45b8-84f7-98c79d80cb9f None admin] RBAC: Authorizing `identity:create_grant(project_id=cd271be4aaeb4efab5ef86157a14122b, group_id=394f7a2fceff4e9cb7c17083c18e2f5f, role_id=fc5d6a08fc2048f19c14f691e35cd41c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:56.637552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8f04c-df45-4ca0-a2fc-34d95ecc039f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:56.637940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8f04c-df45-4ca0-a2fc-34d95ecc039f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:56.638792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88c6f6-428c-45b8-84f7-98c79d80cb9f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:56.639776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 190/394] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:56.647654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc908ce-7761-4ea4-8851-6cd14867aa14 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:56.648035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc908ce-7761-4ea4-8851-6cd14867aa14 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:56.648390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc908ce-7761-4ea4-8851-6cd14867aa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:56.665940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dfc908ce-7761-4ea4-8851-6cd14867aa14 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:56.667403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dfc908ce-7761-4ea4-8851-6cd14867aa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:56.723271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 205/395] 10.222.0.41 () {58 vars in 1442 bytes} [Sat May 28 03:16:56 2022] PUT /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b/groups/394f7a2fceff4e9cb7c17083c18e2f5f/roles/fc5d6a08fc2048f19c14f691e35cd41c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:56.748932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 191/396] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:56.772335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88c6f6-428c-45b8-84f7-98c79d80cb9f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:56.955593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88c6f6-428c-45b8-84f7-98c79d80cb9f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:57.010570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f6be1-c5f9-4dd9-b678-5d96eedb49c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:57.011390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6be1-c5f9-4dd9-b678-5d96eedb49c9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.011675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6be1-c5f9-4dd9-b678-5d96eedb49c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.012194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6be1-c5f9-4dd9-b678-5d96eedb49c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.025495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f6be1-c5f9-4dd9-b678-5d96eedb49c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-655f6be1-c5f9-4dd9-b678-5d96eedb49c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:57.034317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:57.035281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:57.036065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 206/397] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:57.043298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc908ce-7761-4ea4-8851-6cd14867aa14 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:57.156985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc908ce-7761-4ea4-8851-6cd14867aa14 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:57.158589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:57.169754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c0561-4d7b-4543-8b59-654199befd04 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.170012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c0561-4d7b-4543-8b59-654199befd04 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.170278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c0561-4d7b-4543-8b59-654199befd04 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.171902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 207/398] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:57.177918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91468f-de82-4680-9aca-f45a4ca0aa76 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.178469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91468f-de82-4680-9aca-f45a4ca0aa76 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.182102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91468f-de82-4680-9aca-f45a4ca0aa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.184960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:57.196622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4b91468f-de82-4680-9aca-f45a4ca0aa76 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:57.198139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4b91468f-de82-4680-9aca-f45a4ca0aa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:57.218337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4fe3a-6dc4-4e56-aaf4-42dce30e95ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:57.220539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4fe3a-6dc4-4e56-aaf4-42dce30e95ff None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.221020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4fe3a-6dc4-4e56-aaf4-42dce30e95ff None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.222573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4fe3a-6dc4-4e56-aaf4-42dce30e95ff None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.223840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4fe3a-6dc4-4e56-aaf4-42dce30e95ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:57.229725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4fe3a-6dc4-4e56-aaf4-42dce30e95ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:57.231415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4fe3a-6dc4-4e56-aaf4-42dce30e95ff None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:57.244477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 192/399] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:57.248425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b275869-5f3e-4113-89f2-0db9b4fa19b9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b275869-5f3e-4113-89f2-0db9b4fa19b9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.250615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b275869-5f3e-4113-89f2-0db9b4fa19b9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.252212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 193/400] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:57.255688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b275869-5f3e-4113-89f2-0db9b4fa19b9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:57.306443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 208/401] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:57.308433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef135a75-be51-47ef-87fe-2d600831f9eb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.308885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef135a75-be51-47ef-87fe-2d600831f9eb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.309535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef135a75-be51-47ef-87fe-2d600831f9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.331327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ef135a75-be51-47ef-87fe-2d600831f9eb None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:57.332779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ef135a75-be51-47ef-87fe-2d600831f9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:57.371114 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b275869-5f3e-4113-89f2-0db9b4fa19b9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:57.409808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc32a24-3e92-4cc6-ad81-06b10da83d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:57.410663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc32a24-3e92-4cc6-ad81-06b10da83d8b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.410938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc32a24-3e92-4cc6-ad81-06b10da83d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.411206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc32a24-3e92-4cc6-ad81-06b10da83d8b None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.412641 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc32a24-3e92-4cc6-ad81-06b10da83d8b None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:57.413944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc32a24-3e92-4cc6-ad81-06b10da83d8b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:57.415873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 194/402] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:57 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:57.420569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc32a24-3e92-4cc6-ad81-06b10da83d8b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:57.426739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 209/403] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:57.428387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef135a75-be51-47ef-87fe-2d600831f9eb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:57.461508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:57.520648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:57.557236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc32a24-3e92-4cc6-ad81-06b10da83d8b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:57.570064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef135a75-be51-47ef-87fe-2d600831f9eb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:57.601350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba61ba-2b40-4d63-9fe6-8570835916ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:57.602638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba61ba-2b40-4d63-9fe6-8570835916ef None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.603325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba61ba-2b40-4d63-9fe6-8570835916ef None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.603789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba61ba-2b40-4d63-9fe6-8570835916ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.613499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0586afa-c5c2-4dcd-a5a8-2022a45152f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:57.614817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586afa-c5c2-4dcd-a5a8-2022a45152f1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.615196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586afa-c5c2-4dcd-a5a8-2022a45152f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.615565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586afa-c5c2-4dcd-a5a8-2022a45152f1 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.616962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0586afa-c5c2-4dcd-a5a8-2022a45152f1 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:57.617832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba61ba-2b40-4d63-9fe6-8570835916ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:57.618763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0586afa-c5c2-4dcd-a5a8-2022a45152f1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:57.625313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 210/404] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:57 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:57.625313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0586afa-c5c2-4dcd-a5a8-2022a45152f1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-55ba61ba-2b40-4d63-9fe6-8570835916ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:57.630816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:57.631673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:57.632487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 195/405] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:57.633157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba61ba-2b40-4d63-9fe6-8570835916ef None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:57.735319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:57.738465 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0586afa-c5c2-4dcd-a5a8-2022a45152f1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:57.751624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba61ba-2b40-4d63-9fe6-8570835916ef None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:57.769642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:57.776156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81154b-1e72-486c-b840-1c3161fb1a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:57.777239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81154b-1e72-486c-b840-1c3161fb1a0d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.777748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81154b-1e72-486c-b840-1c3161fb1a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.778636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81154b-1e72-486c-b840-1c3161fb1a0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.788744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b55011-a8d9-4015-86da-10c6b0cb1b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:57.789655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55011-a8d9-4015-86da-10c6b0cb1b58 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:57.789937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55011-a8d9-4015-86da-10c6b0cb1b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:57.791595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b55011-a8d9-4015-86da-10c6b0cb1b58 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e/groups/394f7a2fceff4e9cb7c17083c18e2f5f/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:57.792416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81154b-1e72-486c-b840-1c3161fb1a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:57.792543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b55011-a8d9-4015-86da-10c6b0cb1b58 None admin] RBAC: Authorizing `identity:create_grant(project_id=7998a56b1396402ab848761300b3785e, group_id=394f7a2fceff4e9cb7c17083c18e2f5f, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:57.794128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b55011-a8d9-4015-86da-10c6b0cb1b58 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-dd81154b-1e72-486c-b840-1c3161fb1a0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:57.801393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:57.802346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:57.803110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 211/406] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:16:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:57.803757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81154b-1e72-486c-b840-1c3161fb1a0d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:57.873183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 196/407] 10.222.0.41 () {58 vars in 1442 bytes} [Sat May 28 03:16:57 2022] PUT /identity/v3/projects/7998a56b1396402ab848761300b3785e/groups/394f7a2fceff4e9cb7c17083c18e2f5f/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:57.875145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b55011-a8d9-4015-86da-10c6b0cb1b58 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:57.968196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81154b-1e72-486c-b840-1c3161fb1a0d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:58.039921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:58.041329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.042000 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.043818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.044821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:58.045560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:58.048045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:58.066965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 212/408] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:58.073052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:58.097615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b55011-a8d9-4015-86da-10c6b0cb1b58 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:58.148316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:58.148991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.149362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.149705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.150460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:58.150768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:58.152734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:58.168877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:58.171609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 197/409] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:58.176994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:58.191239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:58.227814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e77c50-c534-4ebb-ae73-7e34599c226d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:58.303286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6911b59-6f07-4588-9e31-3bee790dacd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:58.305933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6911b59-6f07-4588-9e31-3bee790dacd1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.307444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6911b59-6f07-4588-9e31-3bee790dacd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.308637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6911b59-6f07-4588-9e31-3bee790dacd1 None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.311282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6911b59-6f07-4588-9e31-3bee790dacd1 None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:58.314786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6911b59-6f07-4588-9e31-3bee790dacd1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:58.316892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 213/410] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:58 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:58.321829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6911b59-6f07-4588-9e31-3bee790dacd1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:58.354856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2c8bf-80db-49a2-bf42-cc26d2b9b861 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:58.401380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291089d-9a0d-411d-8053-a07776970223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:58.402660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291089d-9a0d-411d-8053-a07776970223 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.403197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291089d-9a0d-411d-8053-a07776970223 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.403821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291089d-9a0d-411d-8053-a07776970223 None admin] PATH_INFO: `/v3/groups/bf393a79636f4a40a9039e120d3d0162` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.408376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291089d-9a0d-411d-8053-a07776970223 None admin] RBAC: Authorizing `identity:get_group(group_id=bf393a79636f4a40a9039e120d3d0162)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:58.410334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291089d-9a0d-411d-8053-a07776970223 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:58.412561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 198/411] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:16:58 2022] GET /identity/v3/groups/bf393a79636f4a40a9039e120d3d0162 => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:58.417501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291089d-9a0d-411d-8053-a07776970223 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:58.458250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6911b59-6f07-4588-9e31-3bee790dacd1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:58.478399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:58.507005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839124-bdeb-4943-b27f-b0d6058197a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:58.507874 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839124-bdeb-4943-b27f-b0d6058197a8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.508161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839124-bdeb-4943-b27f-b0d6058197a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.508440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839124-bdeb-4943-b27f-b0d6058197a8 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.509490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:58.509696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b839124-bdeb-4943-b27f-b0d6058197a8 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:58.511105 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b839124-bdeb-4943-b27f-b0d6058197a8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:58.517697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 214/412] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:58 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:58.526880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839124-bdeb-4943-b27f-b0d6058197a8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:58.545411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291089d-9a0d-411d-8053-a07776970223 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:58.587234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de195d73-500f-4675-ae10-2960b30689f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:58.588031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de195d73-500f-4675-ae10-2960b30689f8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.588454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de195d73-500f-4675-ae10-2960b30689f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.588876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de195d73-500f-4675-ae10-2960b30689f8 None admin] PATH_INFO: `/v3/projects/cc9d398b29a149a4ab99273f43b67fb2` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.590109 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de195d73-500f-4675-ae10-2960b30689f8 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9d398b29a149a4ab99273f43b67fb2)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:58.592086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de195d73-500f-4675-ae10-2960b30689f8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:58.594135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 199/413] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:58 2022] GET /identity/v3/projects/cc9d398b29a149a4ab99273f43b67fb2 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:16:58.596219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13924a-038d-4754-8bb1-0b51c1d5d637 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.596652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13924a-038d-4754-8bb1-0b51c1d5d637 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.597239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13924a-038d-4754-8bb1-0b51c1d5d637 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.601182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 200/414] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:16:58.602131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13924a-038d-4754-8bb1-0b51c1d5d637 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:58.661052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839124-bdeb-4943-b27f-b0d6058197a8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:58.704446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970b221-7f1c-49c2-835e-e65d72636e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:58.706152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b221-7f1c-49c2-835e-e65d72636e3f None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.706671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b221-7f1c-49c2-835e-e65d72636e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.707218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970b221-7f1c-49c2-835e-e65d72636e3f None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e/groups/394f7a2fceff4e9cb7c17083c18e2f5f/roles/fc5d6a08fc2048f19c14f691e35cd41c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.710810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970b221-7f1c-49c2-835e-e65d72636e3f None admin] RBAC: Authorizing `identity:create_grant(project_id=7998a56b1396402ab848761300b3785e, group_id=394f7a2fceff4e9cb7c17083c18e2f5f, role_id=fc5d6a08fc2048f19c14f691e35cd41c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:58.713534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970b221-7f1c-49c2-835e-e65d72636e3f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:58.723567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13924a-038d-4754-8bb1-0b51c1d5d637 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:58.758674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91993e4-c03d-404c-a994-e99a489b4ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:58.759556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91993e4-c03d-404c-a994-e99a489b4ba4 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.759839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91993e4-c03d-404c-a994-e99a489b4ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.760214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91993e4-c03d-404c-a994-e99a489b4ba4 None admin] PATH_INFO: `/v3/projects/cc9d398b29a149a4ab99273f43b67fb2/groups/bf393a79636f4a40a9039e120d3d0162/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.762272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91993e4-c03d-404c-a994-e99a489b4ba4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc9d398b29a149a4ab99273f43b67fb2, group_id=bf393a79636f4a40a9039e120d3d0162, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:58.765326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91993e4-c03d-404c-a994-e99a489b4ba4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:58.789140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 215/415] 10.222.0.41 () {58 vars in 1442 bytes} [Sat May 28 03:16:58 2022] PUT /identity/v3/projects/7998a56b1396402ab848761300b3785e/groups/394f7a2fceff4e9cb7c17083c18e2f5f/roles/fc5d6a08fc2048f19c14f691e35cd41c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:58.789140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee931552-6a19-4e13-b491-61782d04db6f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:58.789140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee931552-6a19-4e13-b491-61782d04db6f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:58.789140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee931552-6a19-4e13-b491-61782d04db6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:58.810601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ee931552-6a19-4e13-b491-61782d04db6f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:58.811976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ee931552-6a19-4e13-b491-61782d04db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:58.863683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:58.870597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 201/416] 10.222.0.41 () {58 vars in 1442 bytes} [Sat May 28 03:16:58 2022] PUT /identity/v3/projects/cc9d398b29a149a4ab99273f43b67fb2/groups/bf393a79636f4a40a9039e120d3d0162/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:16:58.898606 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 216/417] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:16:58.922659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91993e4-c03d-404c-a994-e99a489b4ba4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:58.925756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:59.059708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91993e4-c03d-404c-a994-e99a489b4ba4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:59.100331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ae420-171a-4fdf-91ef-7190c056029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:59.101978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae420-171a-4fdf-91ef-7190c056029e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:59.102553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae420-171a-4fdf-91ef-7190c056029e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:59.103035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ae420-171a-4fdf-91ef-7190c056029e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:59.115684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ae420-171a-4fdf-91ef-7190c056029e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-477ae420-171a-4fdf-91ef-7190c056029e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:59.125065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:59.126074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:16:59.126893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:16:59.130029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 202/418] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:16:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:59.136961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee931552-6a19-4e13-b491-61782d04db6f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:59.252760 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee931552-6a19-4e13-b491-61782d04db6f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:59.294518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1dc29-fd81-4675-986f-67bf33e22146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:59.295767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1dc29-fd81-4675-986f-67bf33e22146 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:59.296039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1dc29-fd81-4675-986f-67bf33e22146 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:59.296320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1dc29-fd81-4675-986f-67bf33e22146 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:59.297080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1dc29-fd81-4675-986f-67bf33e22146 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:59.297415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1dc29-fd81-4675-986f-67bf33e22146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:59.298814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1dc29-fd81-4675-986f-67bf33e22146 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:59.312766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 217/419] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:16:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:59.317837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ae420-171a-4fdf-91ef-7190c056029e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:59.323059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4db04-1783-4f22-bf8b-a297b661f6e5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:59.323355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4db04-1783-4f22-bf8b-a297b661f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:59.323807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a4db04-1783-4f22-bf8b-a297b661f6e5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:59.325420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 218/420] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:16:59.331065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85f7cc-fe50-4ac1-82b7-f00b453d4029 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:59.331355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85f7cc-fe50-4ac1-82b7-f00b453d4029 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:59.331634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85f7cc-fe50-4ac1-82b7-f00b453d4029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:59.355916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2e85f7cc-fe50-4ac1-82b7-f00b453d4029 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:16:59.357146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2e85f7cc-fe50-4ac1-82b7-f00b453d4029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:16:59.415603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:59.449748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 219/421] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:16:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:16:59.454992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:59.494046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85f7cc-fe50-4ac1-82b7-f00b453d4029 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:59.515696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ae420-171a-4fdf-91ef-7190c056029e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:59.559449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955f7e-42eb-4e06-a61e-d6dda256575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:59.560100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955f7e-42eb-4e06-a61e-d6dda256575a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:59.560436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955f7e-42eb-4e06-a61e-d6dda256575a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:59.560715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955f7e-42eb-4e06-a61e-d6dda256575a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:59.639695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c955f7e-42eb-4e06-a61e-d6dda256575a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:59.641220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c955f7e-42eb-4e06-a61e-d6dda256575a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:59.713657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:59.723271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85f7cc-fe50-4ac1-82b7-f00b453d4029 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:59.746947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 203/422] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:16:59.754272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955f7e-42eb-4e06-a61e-d6dda256575a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:59.773836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3247e-9cbc-4b3d-a0c3-abf315360e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:59.774662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3247e-9cbc-4b3d-a0c3-abf315360e3c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:59.774948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3247e-9cbc-4b3d-a0c3-abf315360e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:59.775230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3247e-9cbc-4b3d-a0c3-abf315360e3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:59.786250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3247e-9cbc-4b3d-a0c3-abf315360e3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2cf3247e-9cbc-4b3d-a0c3-abf315360e3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:16:59.794492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:16:59.795628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:16:59.796624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:16:59.800515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 220/423] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:16:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:16:59.801725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3247e-9cbc-4b3d-a0c3-abf315360e3c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:59.835141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:16:59.916738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3247e-9cbc-4b3d-a0c3-abf315360e3c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:16:59.952777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b2915-d758-4049-918b-89c389219953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:16:59.953641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2915-d758-4049-918b-89c389219953 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:16:59.954025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2915-d758-4049-918b-89c389219953 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:16:59.954403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b2915-d758-4049-918b-89c389219953 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:16:59.955262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b2915-d758-4049-918b-89c389219953 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:16:59.955693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b2915-d758-4049-918b-89c389219953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:16:59.957280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b2915-d758-4049-918b-89c389219953 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:16:59.970613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 221/424] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:16:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:16:59.972025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b2915-d758-4049-918b-89c389219953 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:16:59.989857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955f7e-42eb-4e06-a61e-d6dda256575a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:00.055980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7a45f-b14f-4741-9765-1ab0e6539ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:00.056841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7a45f-b14f-4741-9765-1ab0e6539ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:00.057141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7a45f-b14f-4741-9765-1ab0e6539ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:00.057424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7a45f-b14f-4741-9765-1ab0e6539ba2 None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:00.058729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7a45f-b14f-4741-9765-1ab0e6539ba2 None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:00.060245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7a45f-b14f-4741-9765-1ab0e6539ba2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:00.062926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 204/425] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:16:59 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:00.068173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7a45f-b14f-4741-9765-1ab0e6539ba2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:00.103339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:00.118712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b2915-d758-4049-918b-89c389219953 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:00.119240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:00.175692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6843e23-e765-4a44-8b78-c8a72356c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:00.176335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6843e23-e765-4a44-8b78-c8a72356c32a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:00.176616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6843e23-e765-4a44-8b78-c8a72356c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:00.176892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6843e23-e765-4a44-8b78-c8a72356c32a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:00.211829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7a45f-b14f-4741-9765-1ab0e6539ba2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:00.256362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7f0ec-55b4-474a-8b6b-6f6c2e5c4b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:00.257356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7f0ec-55b4-474a-8b6b-6f6c2e5c4b3b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:00.257641 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7f0ec-55b4-474a-8b6b-6f6c2e5c4b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:00.258020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7f0ec-55b4-474a-8b6b-6f6c2e5c4b3b None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:00.261058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7f0ec-55b4-474a-8b6b-6f6c2e5c4b3b None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:00.262664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7f0ec-55b4-474a-8b6b-6f6c2e5c4b3b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:00.268950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 205/426] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:17:00 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:00.269980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7f0ec-55b4-474a-8b6b-6f6c2e5c4b3b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:00.286077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6843e23-e765-4a44-8b78-c8a72356c32a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:00.287886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6843e23-e765-4a44-8b78-c8a72356c32a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:00.421498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:00.429913 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7f0ec-55b4-474a-8b6b-6f6c2e5c4b3b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:00.469468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:00.470602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b5a5d-fbb2-4f8c-88cf-058ff0353715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:00.471267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b5a5d-fbb2-4f8c-88cf-058ff0353715 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:00.471557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b5a5d-fbb2-4f8c-88cf-058ff0353715 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:00.472099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b5a5d-fbb2-4f8c-88cf-058ff0353715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:00.472937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b5a5d-fbb2-4f8c-88cf-058ff0353715 None admin] RBAC: Adding query filter params (group.id=394f7a2fceff4e9cb7c17083c18e2f5f, role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=cd271be4aaeb4efab5ef86157a14122b) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:00.473284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b5a5d-fbb2-4f8c-88cf-058ff0353715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:00.475457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b5a5d-fbb2-4f8c-88cf-058ff0353715 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:00.489621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 222/427] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:16:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 03:17:00.490586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 206/428] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:17:00 2022] GET /identity/v3/role_assignments?group.id=394f7a2fceff4e9cb7c17083c18e2f5f&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:00.495557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6843e23-e765-4a44-8b78-c8a72356c32a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:00.741290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b206118-8103-49cc-9c2a-75b3599ee5af None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:00.741609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b206118-8103-49cc-9c2a-75b3599ee5af None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:00.741782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b206118-8103-49cc-9c2a-75b3599ee5af None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:00.744072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 207/429] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:00.750766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2544-c97d-40b7-abb7-b07c2d1b9f11 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:00.751059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2544-c97d-40b7-abb7-b07c2d1b9f11 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:00.751343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f2544-c97d-40b7-abb7-b07c2d1b9f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:00.763109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6843e23-e765-4a44-8b78-c8a72356c32a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:00.788179 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-813f2544-c97d-40b7-abb7-b07c2d1b9f11 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:00.788603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-813f2544-c97d-40b7-abb7-b07c2d1b9f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:00.834388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667e8ef-b54d-484f-ba60-097858daa65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:00.835266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e8ef-b54d-484f-ba60-097858daa65e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:00.835552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e8ef-b54d-484f-ba60-097858daa65e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:00.835888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667e8ef-b54d-484f-ba60-097858daa65e None admin] PATH_INFO: `/v3/projects/cd271be4aaeb4efab5ef86157a14122b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:00.837029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667e8ef-b54d-484f-ba60-097858daa65e None admin] RBAC: Authorizing `identity:get_project(project_id=cd271be4aaeb4efab5ef86157a14122b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:00.838516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667e8ef-b54d-484f-ba60-097858daa65e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:00.844694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 223/430] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:17:00 2022] GET /identity/v3/projects/cd271be4aaeb4efab5ef86157a14122b => generated 298 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:00.846265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667e8ef-b54d-484f-ba60-097858daa65e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:00.922891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 208/431] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:00.947524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f2544-c97d-40b7-abb7-b07c2d1b9f11 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:00.956953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:01.079456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:01.092031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f2544-c97d-40b7-abb7-b07c2d1b9f11 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:01.107274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667e8ef-b54d-484f-ba60-097858daa65e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:01.145748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326cf4-9c2e-4b40-87df-791bd46aab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:01.147236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326cf4-9c2e-4b40-87df-791bd46aab52 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.147739 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326cf4-9c2e-4b40-87df-791bd46aab52 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.148214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326cf4-9c2e-4b40-87df-791bd46aab52 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.157778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf6a75-6826-48c2-a14b-b2d90da873eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:01.158644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf6a75-6826-48c2-a14b-b2d90da873eb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.158931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf6a75-6826-48c2-a14b-b2d90da873eb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.159210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf6a75-6826-48c2-a14b-b2d90da873eb None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.160395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf6a75-6826-48c2-a14b-b2d90da873eb None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:01.161891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf6a75-6826-48c2-a14b-b2d90da873eb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:01.162671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6326cf4-9c2e-4b40-87df-791bd46aab52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:01.163813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 224/432] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:17:00 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:01.170236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf6a75-6826-48c2-a14b-b2d90da873eb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-d6326cf4-9c2e-4b40-87df-791bd46aab52 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:01.172213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:01.173284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 03:17:01.174150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:01.176624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 209/433] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:01.184274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326cf4-9c2e-4b40-87df-791bd46aab52 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:01.283694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf6a75-6826-48c2-a14b-b2d90da873eb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:01.298199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326cf4-9c2e-4b40-87df-791bd46aab52 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:01.311891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:01.320054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:01.320741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.321032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.329433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.329433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] RBAC: Adding query filter params (group.id=394f7a2fceff4e9cb7c17083c18e2f5f, role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=cd271be4aaeb4efab5ef86157a14122b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:01.329433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:01.329433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:01.332978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 225/434] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:17:01 2022] GET /identity/v3/role_assignments?group.id=394f7a2fceff4e9cb7c17083c18e2f5f&scope.project.id=cd271be4aaeb4efab5ef86157a14122b&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:01.341690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0a454-5a4c-4081-887a-75e11d841b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:01.342455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a454-5a4c-4081-887a-75e11d841b52 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.342748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a454-5a4c-4081-887a-75e11d841b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.344067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0a454-5a4c-4081-887a-75e11d841b52 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.344067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0a454-5a4c-4081-887a-75e11d841b52 None admin] RBAC: Adding query filter params (name=member) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:01.344326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0a454-5a4c-4081-887a-75e11d841b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:01.345831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0a454-5a4c-4081-887a-75e11d841b52 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:01.357586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 210/435] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:17:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:01.363089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:01.478700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abe964-fa11-4571-a233-e2df297046ab None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:01.516922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e6a58-f757-4123-bfdf-47bdcf26e3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:01.517613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e6a58-f757-4123-bfdf-47bdcf26e3dd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.517903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e6a58-f757-4123-bfdf-47bdcf26e3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.518187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e6a58-f757-4123-bfdf-47bdcf26e3dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.545922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d77ceb-80ef-4b9a-8432-343c497cdb22 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.546265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d77ceb-80ef-4b9a-8432-343c497cdb22 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.546500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d77ceb-80ef-4b9a-8432-343c497cdb22 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.548905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 211/436] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:01.555094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9172-165a-4bf4-9d1e-dbe61d83115e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.555412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9172-165a-4bf4-9d1e-dbe61d83115e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.555587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d9172-165a-4bf4-9d1e-dbe61d83115e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.582164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-921d9172-165a-4bf4-9d1e-dbe61d83115e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:01.583507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-921d9172-165a-4bf4-9d1e-dbe61d83115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:01.592491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:01.594922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:01.606188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e6a58-f757-4123-bfdf-47bdcf26e3dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:01.610154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e6a58-f757-4123-bfdf-47bdcf26e3dd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:01.683026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 212/437] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:01.707294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d9172-165a-4bf4-9d1e-dbe61d83115e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:01.744903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 226/438] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 03:17:01.746421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c8c52-800d-4eda-97cf-a6faa75ef831 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.746715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c8c52-800d-4eda-97cf-a6faa75ef831 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.746993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c8c52-800d-4eda-97cf-a6faa75ef831 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.749436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 227/439] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:01.750965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c8c52-800d-4eda-97cf-a6faa75ef831 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:01.831265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d9172-165a-4bf4-9d1e-dbe61d83115e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:01.863305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:01.868017 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5680346-5d15-4440-9ae2-b102d0307998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:01.868827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5680346-5d15-4440-9ae2-b102d0307998 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.869134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5680346-5d15-4440-9ae2-b102d0307998 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.869422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5680346-5d15-4440-9ae2-b102d0307998 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.872720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c8c52-800d-4eda-97cf-a6faa75ef831 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:01.879944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5680346-5d15-4440-9ae2-b102d0307998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-d5680346-5d15-4440-9ae2-b102d0307998 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:01.888058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:01.889022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 213/440] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:01.897155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531121-f15a-471a-bf3a-eb5bf3681b15 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.898921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531121-f15a-471a-bf3a-eb5bf3681b15 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.898921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7531121-f15a-471a-bf3a-eb5bf3681b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.901042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:01.911155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba2f92-5dac-4569-80ae-f92c250be63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:01.912017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba2f92-5dac-4569-80ae-f92c250be63f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:01.912299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba2f92-5dac-4569-80ae-f92c250be63f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:01.912572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba2f92-5dac-4569-80ae-f92c250be63f None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:01.913753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba2f92-5dac-4569-80ae-f92c250be63f None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:01.914671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a7531121-f15a-471a-bf3a-eb5bf3681b15 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:01.915178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba2f92-5dac-4569-80ae-f92c250be63f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:01.915866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a7531121-f15a-471a-bf3a-eb5bf3681b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:01.916845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 228/441] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:17:01 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:01.917633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba2f92-5dac-4569-80ae-f92c250be63f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:01.983896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 214/442] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:01.984721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531121-f15a-471a-bf3a-eb5bf3681b15 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.017258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba2f92-5dac-4569-80ae-f92c250be63f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.050878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.051529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.051809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.052079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.052803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:02.053117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.054477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.062570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 229/443] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:02.063359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.080051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7531121-f15a-471a-bf3a-eb5bf3681b15 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.086591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:02.119410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4d05-7f97-41be-95ae-1e4b87082a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.120181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4d05-7f97-41be-95ae-1e4b87082a59 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.120463 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4d05-7f97-41be-95ae-1e4b87082a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.120738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb4d05-7f97-41be-95ae-1e4b87082a59 None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.121940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb4d05-7f97-41be-95ae-1e4b87082a59 None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.123394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb4d05-7f97-41be-95ae-1e4b87082a59 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.124982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 215/444] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:17:01 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:02.125780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4d05-7f97-41be-95ae-1e4b87082a59 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.128311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:02.163154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eee89-658d-41c8-8cc6-6a91ba25031a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.196720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681d3f3-2622-457a-b110-8cb6961a445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.197543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681d3f3-2622-457a-b110-8cb6961a445d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.197830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681d3f3-2622-457a-b110-8cb6961a445d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.198109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681d3f3-2622-457a-b110-8cb6961a445d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.208084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681d3f3-2622-457a-b110-8cb6961a445d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6681d3f3-2622-457a-b110-8cb6961a445d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:02.215790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:02.217216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:02.217993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 230/445] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:02.218777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681d3f3-2622-457a-b110-8cb6961a445d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.228756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb4d05-7f97-41be-95ae-1e4b87082a59 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.261330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e83fb-ecb0-48d3-958b-cbb860445b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.261960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e83fb-ecb0-48d3-958b-cbb860445b9f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.262240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e83fb-ecb0-48d3-958b-cbb860445b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.262512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e83fb-ecb0-48d3-958b-cbb860445b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.298553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:02.315600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681d3f3-2622-457a-b110-8cb6961a445d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.328546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e83fb-ecb0-48d3-958b-cbb860445b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.330038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e83fb-ecb0-48d3-958b-cbb860445b9f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.331802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:02.350294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.350942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.351225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.351504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.352328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] RBAC: Adding query filter params (group.id=394f7a2fceff4e9cb7c17083c18e2f5f, role.id=8e1fb3d93ff549a2899c16dd1130e587, scope.project.id=7998a56b1396402ab848761300b3785e) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:02.352644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.354094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.362622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 231/446] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/role_assignments?group.id=394f7a2fceff4e9cb7c17083c18e2f5f&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=8e1fb3d93ff549a2899c16dd1130e587 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:02.364581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.451888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 216/447] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:02.457423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e83fb-ecb0-48d3-958b-cbb860445b9f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.485071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0459c0-dafa-4e0e-a420-a0720eb2bf98 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.528443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.529716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.530003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.530281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.531057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:02.531355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.532759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.541853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 232/448] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:02.547522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.560892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:02.579319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e83fb-ecb0-48d3-958b-cbb860445b9f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.580887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:02.621310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ed784-2808-47cf-9d37-3734467ef050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.622182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ed784-2808-47cf-9d37-3734467ef050 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.622520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ed784-2808-47cf-9d37-3734467ef050 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.622832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ed784-2808-47cf-9d37-3734467ef050 None admin] PATH_INFO: `/v3/projects/7998a56b1396402ab848761300b3785e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.624135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ed784-2808-47cf-9d37-3734467ef050 None admin] RBAC: Authorizing `identity:get_project(project_id=7998a56b1396402ab848761300b3785e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.625707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ed784-2808-47cf-9d37-3734467ef050 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.628266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 217/449] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/projects/7998a56b1396402ab848761300b3785e => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:02.633311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ed784-2808-47cf-9d37-3734467ef050 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.669767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbef446-1a7c-4d01-9a68-30cd10fe1a85 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.710572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034524ae-dccf-4183-a836-f7c2e8a44c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.711243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034524ae-dccf-4183-a836-f7c2e8a44c01 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.711534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034524ae-dccf-4183-a836-f7c2e8a44c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.711808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034524ae-dccf-4183-a836-f7c2e8a44c01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.760566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ed784-2808-47cf-9d37-3734467ef050 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.786906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034524ae-dccf-4183-a836-f7c2e8a44c01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.788391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034524ae-dccf-4183-a836-f7c2e8a44c01 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.793856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e58d-353d-4dab-9dd2-5e5ead5b8b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.794720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e58d-353d-4dab-9dd2-5e5ead5b8b05 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.794996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e58d-353d-4dab-9dd2-5e5ead5b8b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.795264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e58d-353d-4dab-9dd2-5e5ead5b8b05 None admin] PATH_INFO: `/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.796423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338e58d-353d-4dab-9dd2-5e5ead5b8b05 None admin] RBAC: Authorizing `identity:get_group(group_id=394f7a2fceff4e9cb7c17083c18e2f5f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.797884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338e58d-353d-4dab-9dd2-5e5ead5b8b05 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.798612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:02.799503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 218/450] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/groups/394f7a2fceff4e9cb7c17083c18e2f5f => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:02.805037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e58d-353d-4dab-9dd2-5e5ead5b8b05 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.845318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:02.887071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 233/451] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 03:17:02.893284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034524ae-dccf-4183-a836-f7c2e8a44c01 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:02.902701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e58d-353d-4dab-9dd2-5e5ead5b8b05 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:02.936820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:02.937505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:02.937790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:02.938066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:02.938888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] RBAC: Adding query filter params (group.id=394f7a2fceff4e9cb7c17083c18e2f5f, role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=7998a56b1396402ab848761300b3785e) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:02.939196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:02.940640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:02.946428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 219/452] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/role_assignments?group.id=394f7a2fceff4e9cb7c17083c18e2f5f&scope.project.id=7998a56b1396402ab848761300b3785e&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:02.998351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034524ae-dccf-4183-a836-f7c2e8a44c01 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:03.009802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:03.036557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be051ff5-8393-4b03-8af2-a03976759ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:03.037446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be051ff5-8393-4b03-8af2-a03976759ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:03.037728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be051ff5-8393-4b03-8af2-a03976759ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:03.038001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be051ff5-8393-4b03-8af2-a03976759ab9 None admin] PATH_INFO: `/v3/projects/cc9d398b29a149a4ab99273f43b67fb2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:03.039225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be051ff5-8393-4b03-8af2-a03976759ab9 None admin] RBAC: Authorizing `identity:get_project(project_id=cc9d398b29a149a4ab99273f43b67fb2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:03.040674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be051ff5-8393-4b03-8af2-a03976759ab9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:03.042727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 234/453] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:17:02 2022] GET /identity/v3/projects/cc9d398b29a149a4ab99273f43b67fb2 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:03.048642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:03.151995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afc21b-652b-497b-af21-6d3de5228f63 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:03.153573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:03.185057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a9278-060d-4c4f-b75f-7deee05ac1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:03.186479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9278-060d-4c4f-b75f-7deee05ac1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:03.186759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9278-060d-4c4f-b75f-7deee05ac1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:03.187037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9278-060d-4c4f-b75f-7deee05ac1b8 None admin] PATH_INFO: `/v3/groups/bf393a79636f4a40a9039e120d3d0162` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:03.188204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a9278-060d-4c4f-b75f-7deee05ac1b8 None admin] RBAC: Authorizing `identity:get_group(group_id=bf393a79636f4a40a9039e120d3d0162)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:03.189672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a9278-060d-4c4f-b75f-7deee05ac1b8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:03.191254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 220/454] 10.222.0.41 () {56 vars in 1180 bytes} [Sat May 28 03:17:03 2022] GET /identity/v3/groups/bf393a79636f4a40a9039e120d3d0162 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:03.197040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be051ff5-8393-4b03-8af2-a03976759ab9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:03.296513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be051ff5-8393-4b03-8af2-a03976759ab9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:03.345451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899e99b-c37e-4786-af21-d552f129defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:03.346419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899e99b-c37e-4786-af21-d552f129defd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:03.347109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899e99b-c37e-4786-af21-d552f129defd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:03.347534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899e99b-c37e-4786-af21-d552f129defd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:03.348854 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899e99b-c37e-4786-af21-d552f129defd None admin] RBAC: Adding query filter params (group.id=bf393a79636f4a40a9039e120d3d0162, role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=cc9d398b29a149a4ab99273f43b67fb2) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:03.349375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899e99b-c37e-4786-af21-d552f129defd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:03.351656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899e99b-c37e-4786-af21-d552f129defd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:03.359478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 235/455] 10.222.0.41 () {56 vars in 1376 bytes} [Sat May 28 03:17:03 2022] GET /identity/v3/role_assignments?group.id=bf393a79636f4a40a9039e120d3d0162&scope.project.id=cc9d398b29a149a4ab99273f43b67fb2&role.id=bb0756f8af694633b9743a958eb46e7a => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:05.868251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c663e-2c90-4337-a76e-5ecdb44644c3 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:05.868779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c663e-2c90-4337-a76e-5ecdb44644c3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:05.868779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c663e-2c90-4337-a76e-5ecdb44644c3 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:05.870459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 221/456] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:05.876020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f58706-1d48-4c6f-b7b7-a75339d88996 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:05.876827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f58706-1d48-4c6f-b7b7-a75339d88996 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:05.877413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f58706-1d48-4c6f-b7b7-a75339d88996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:05.905051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f5f58706-1d48-4c6f-b7b7-a75339d88996 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:05.906645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f5f58706-1d48-4c6f-b7b7-a75339d88996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:05.962224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:06.013143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 236/457] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:17:06.062110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c663e-2c90-4337-a76e-5ecdb44644c3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:06.126178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99590-a583-44ac-86c6-8620d4e90090 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.126855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99590-a583-44ac-86c6-8620d4e90090 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.127485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99590-a583-44ac-86c6-8620d4e90090 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.129437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 237/458] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:06.134074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14275379-446d-416c-9853-51bbf74504f6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.134687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14275379-446d-416c-9853-51bbf74504f6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.135168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14275379-446d-416c-9853-51bbf74504f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.170668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-14275379-446d-416c-9853-51bbf74504f6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:06.172085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-14275379-446d-416c-9853-51bbf74504f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:06.200989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c663e-2c90-4337-a76e-5ecdb44644c3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:06.243327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:06.244129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.244410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.244685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.263574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:06.265085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:06.300484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 238/459] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:06.302211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:06.303079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:06.303911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:06.303911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:06.303911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:06.303911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:06.303911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 222/460] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:06.314180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14275379-446d-416c-9853-51bbf74504f6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:06.335338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:06.385717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:06.434667 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14275379-446d-416c-9853-51bbf74504f6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:06.437461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:06.498116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:06.498783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.499069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.499348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.500076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:06.500383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:06.502143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:06.530231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 239/461] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:06.536066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:06.563020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0193ff4-6340-47e2-925d-78ebae8da1f4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:06.604794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:06.605616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.606374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.606659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.619749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:06.621499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:06.641658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:06.642497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:06.643312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:06.643312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:06.643312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:06.643312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:06.643513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 223/462] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:06.654604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:06.746664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80c94b-ad31-44db-9aa1-51131f25060c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:06.768457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:06.801305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a0eb4-a876-4350-948c-0e229d87429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:06.801858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8ae60-0956-4fea-85e5-6791ba1b724e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:06.802140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0eb4-a876-4350-948c-0e229d87429e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.802431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0eb4-a876-4350-948c-0e229d87429e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.802709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a0eb4-a876-4350-948c-0e229d87429e None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.803412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a0eb4-a876-4350-948c-0e229d87429e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:06.804886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a0eb4-a876-4350-948c-0e229d87429e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:06.836996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c55f8-88af-4f7c-9ee2-5a95a39b359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:06.837684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c55f8-88af-4f7c-9ee2-5a95a39b359a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.837970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c55f8-88af-4f7c-9ee2-5a95a39b359a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.838247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c55f8-88af-4f7c-9ee2-5a95a39b359a None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.838964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c55f8-88af-4f7c-9ee2-5a95a39b359a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:06.839279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c55f8-88af-4f7c-9ee2-5a95a39b359a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:06.840650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c55f8-88af-4f7c-9ee2-5a95a39b359a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:06.859887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 224/463] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:06.861501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b742c-b168-4719-bfc9-3718ec50e2f6 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.861718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b742c-b168-4719-bfc9-3718ec50e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.861868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b742c-b168-4719-bfc9-3718ec50e2f6 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.863244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 225/464] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:06.865502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb62e3-2aad-4c02-8fb4-05e4c2af17e7 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.865791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb62e3-2aad-4c02-8fb4-05e4c2af17e7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.866070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb62e3-2aad-4c02-8fb4-05e4c2af17e7 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.867507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 226/465] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:06.868016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb62e3-2aad-4c02-8fb4-05e4c2af17e7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:06.908101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 240/466] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:17:06 2022] POST /identity/v3/users => generated 246 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:06.908101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dec491-1bb6-4610-8083-4149ddb4af40 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:06.908101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dec491-1bb6-4610-8083-4149ddb4af40 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:06.908691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dec491-1bb6-4610-8083-4149ddb4af40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:06.929984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-14dec491-1bb6-4610-8083-4149ddb4af40 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:06.931593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-14dec491-1bb6-4610-8083-4149ddb4af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:06.949856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:07.019475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb62e3-2aad-4c02-8fb4-05e4c2af17e7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:07.033636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 241/467] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:07.035261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3492e9-4ea0-4da8-9454-844eee015dc6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.035982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3492e9-4ea0-4da8-9454-844eee015dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.036543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3492e9-4ea0-4da8-9454-844eee015dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.054842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0c3492e9-4ea0-4da8-9454-844eee015dc6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:07.056344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0c3492e9-4ea0-4da8-9454-844eee015dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:07.062238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7639b-d0d5-4ab0-b2d1-f542388d500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:07.063034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7639b-d0d5-4ab0-b2d1-f542388d500c None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.063418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7639b-d0d5-4ab0-b2d1-f542388d500c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.063792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7639b-d0d5-4ab0-b2d1-f542388d500c None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.064599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7639b-d0d5-4ab0-b2d1-f542388d500c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:07.066199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7639b-d0d5-4ab0-b2d1-f542388d500c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:07.076352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:07.141319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 242/468] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:07.142009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3492e9-4ea0-4da8-9454-844eee015dc6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:07.175504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 227/469] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:17:06 2022] POST /identity/v3/users => generated 248 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:07.176292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7639b-d0d5-4ab0-b2d1-f542388d500c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:07.233907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:07.262341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3492e9-4ea0-4da8-9454-844eee015dc6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:07.308796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7639b-d0d5-4ab0-b2d1-f542388d500c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:07.318076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:07.318988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.319285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.319562 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.340473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:07.342060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:07.355258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:07.356115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.356404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.356681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:07.361666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:07.362543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:07.363371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:07.363371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:07.363371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:07.363371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:07.363371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 243/470] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:07.369378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:07.370972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:07.373140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:07.392304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:07.393374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:07.394213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:07.394213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:07.394213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:07.394213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:07.394213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 228/471] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:07.408305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:07.477036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:07.525998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5443d2-87f8-47ad-b094-672d141f5192 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:07.551765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d0754-42b2-4690-833f-d5f85d8f3efa None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:07.559246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:07.570923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9234f13-9ef1-4452-b86e-e2ca2285c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:07.571700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9234f13-9ef1-4452-b86e-e2ca2285c1de None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.572015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9234f13-9ef1-4452-b86e-e2ca2285c1de None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.572293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9234f13-9ef1-4452-b86e-e2ca2285c1de None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.573951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9234f13-9ef1-4452-b86e-e2ca2285c1de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:07.573951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9234f13-9ef1-4452-b86e-e2ca2285c1de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:07.575155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9234f13-9ef1-4452-b86e-e2ca2285c1de None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:07.594825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 244/472] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:07.597338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e3fa2-0003-41e5-9ba9-71a32e6adb50 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.597631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e3fa2-0003-41e5-9ba9-71a32e6adb50 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.597911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e3fa2-0003-41e5-9ba9-71a32e6adb50 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.598999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bcd5b-d3e5-4970-ab51-901c3e6e3654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:07.599125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 245/473] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:07.600142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bcd5b-d3e5-4970-ab51-901c3e6e3654 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.600437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e3fa2-0003-41e5-9ba9-71a32e6adb50 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:07.600802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bcd5b-d3e5-4970-ab51-901c3e6e3654 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.605655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bcd5b-d3e5-4970-ab51-901c3e6e3654 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.607258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bcd5b-d3e5-4970-ab51-901c3e6e3654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:07.607510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bcd5b-d3e5-4970-ab51-901c3e6e3654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:07.611215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bcd5b-d3e5-4970-ab51-901c3e6e3654 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:07.631873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 229/474] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:07.635089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6014f9-f1b9-4e88-a2a3-9852360d61e5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.635089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6014f9-f1b9-4e88-a2a3-9852360d61e5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.635089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6014f9-f1b9-4e88-a2a3-9852360d61e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.652196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6a6014f9-f1b9-4e88-a2a3-9852360d61e5 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:07.653620 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6a6014f9-f1b9-4e88-a2a3-9852360d61e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:07.724469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e3fa2-0003-41e5-9ba9-71a32e6adb50 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:07.733227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 230/475] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:07.733227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6014f9-f1b9-4e88-a2a3-9852360d61e5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:07.758304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:07.787516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcb132-cc69-44ed-bb53-72b542f87688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:07.788416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb132-cc69-44ed-bb53-72b542f87688 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.788864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb132-cc69-44ed-bb53-72b542f87688 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.789318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcb132-cc69-44ed-bb53-72b542f87688 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.790176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcb132-cc69-44ed-bb53-72b542f87688 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:07.791786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcb132-cc69-44ed-bb53-72b542f87688 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:07.832756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6014f9-f1b9-4e88-a2a3-9852360d61e5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:07.869658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fb30b-7690-47d3-83a1-93e96ba85dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:07.870331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb30b-7690-47d3-83a1-93e96ba85dfc None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:07.870615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb30b-7690-47d3-83a1-93e96ba85dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:07.870935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb30b-7690-47d3-83a1-93e96ba85dfc None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:07.871629 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fb30b-7690-47d3-83a1-93e96ba85dfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:07.873126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5fb30b-7690-47d3-83a1-93e96ba85dfc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:07.900421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 246/476] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:17:07 2022] POST /identity/v3/users => generated 249 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:07.901299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcb132-cc69-44ed-bb53-72b542f87688 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:07.965252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 231/477] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:17:07 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:07.969051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:08.044147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcb132-cc69-44ed-bb53-72b542f87688 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:08.083439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:08.084532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:08.085016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:08.085623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:08.086655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:08.088332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:08.128353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.sql.core [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 03:17:08.128353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 28 03:17:08.128353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [parameters: {'id': 'cfb56bd8b373402f9b82632a821ab26a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 28 03:17:08.128353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() May 28 03:17:08.149689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return ref.to_dict() May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 03:17:08.150620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application yield resource May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.rollback() May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.force_reraise() May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise self.value May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application session.commit() May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 03:17:08.151532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._prepare_impl() May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.flush() May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._flush(objects) May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application compat.raise_( May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application flush_context.execute() May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rec.execute(self) May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 03:17:08.152387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application c = connection._execute_20( May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 03:17:08.154120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 03:17:08.156111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [parameters: {'id': 'cfb56bd8b373402f9b82632a821ab26a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:08.158128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 28 03:17:08.160081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:08.167621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 247/478] 10.222.0.41 () {60 vars in 1131 bytes} [Sat May 28 03:17:07 2022] POST /identity/v3/roles => generated 144 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:08.171463 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fb30b-7690-47d3-83a1-93e96ba85dfc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:08.277627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:08.291255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fb30b-7690-47d3-83a1-93e96ba85dfc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:08.331175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cd069-0630-4ec6-b016-f3078536386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:08.331921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cd069-0630-4ec6-b016-f3078536386c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:08.332229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cd069-0630-4ec6-b016-f3078536386c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:08.332485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cd069-0630-4ec6-b016-f3078536386c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:08.343197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cd069-0630-4ec6-b016-f3078536386c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1e7cd069-0630-4ec6-b016-f3078536386c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:08.353082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:08.354048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:08.354834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 232/479] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:08.362540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:08.474251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fdde-8e48-4e9e-8f8e-e9adebd5a9fc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:08.484092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:08.508875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232610ba-c9f2-4f6f-950d-49862f58061a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:08.509579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232610ba-c9f2-4f6f-950d-49862f58061a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:08.509863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232610ba-c9f2-4f6f-950d-49862f58061a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:08.510162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232610ba-c9f2-4f6f-950d-49862f58061a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:08.510888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232610ba-c9f2-4f6f-950d-49862f58061a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:08.511202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232610ba-c9f2-4f6f-950d-49862f58061a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:08.512570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232610ba-c9f2-4f6f-950d-49862f58061a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:08.522017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 248/480] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:09.590845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fba6d-09a5-4f5b-ad73-c9dc789c0f46 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:09.591433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fba6d-09a5-4f5b-ad73-c9dc789c0f46 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:09.591511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fba6d-09a5-4f5b-ad73-c9dc789c0f46 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:09.593173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 233/481] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:09.598585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa485c-0bb8-4c33-ad40-6763a7c314c0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:09.598992 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa485c-0bb8-4c33-ad40-6763a7c314c0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:09.599154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa485c-0bb8-4c33-ad40-6763a7c314c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:09.617978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d8fa485c-0bb8-4c33-ad40-6763a7c314c0 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:09.619196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d8fa485c-0bb8-4c33-ad40-6763a7c314c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:09.704487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 249/482] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:17:09.751597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fba6d-09a5-4f5b-ad73-c9dc789c0f46 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:09.793076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f0bf-0a3c-4aac-9c0d-69b8cd7a0121 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:09.794024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f0bf-0a3c-4aac-9c0d-69b8cd7a0121 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:09.795370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460f0bf-0a3c-4aac-9c0d-69b8cd7a0121 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:09.797692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 250/483] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:09.802691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1ea6-033e-4099-832f-372d17255311 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:09.803206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1ea6-033e-4099-832f-372d17255311 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:09.803747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1ea6-033e-4099-832f-372d17255311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:09.833061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1a1b1ea6-033e-4099-832f-372d17255311 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:09.834491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1a1b1ea6-033e-4099-832f-372d17255311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:09.879989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fba6d-09a5-4f5b-ad73-c9dc789c0f46 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:09.920855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3b9bf-fb41-4a5d-a766-550a1d9a9fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:09.922034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3b9bf-fb41-4a5d-a766-550a1d9a9fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:09.922527 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3b9bf-fb41-4a5d-a766-550a1d9a9fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:09.923007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3b9bf-fb41-4a5d-a766-550a1d9a9fc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:09.935969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3b9bf-fb41-4a5d-a766-550a1d9a9fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-2fe3b9bf-fb41-4a5d-a766-550a1d9a9fc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:09.945534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:09.946425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:09.947209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:09.950049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 234/484] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:09.955623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3b9bf-fb41-4a5d-a766-550a1d9a9fc3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:09.977547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:10.001464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 251/485] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:17:10.069276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b1ea6-033e-4099-832f-372d17255311 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:10.105884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3b9bf-fb41-4a5d-a766-550a1d9a9fc3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:10.151062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:10.152057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:10.152548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:10.153030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:10.154751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:10.155531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:10.156640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:10.158280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:10.168448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 235/486] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:10.180308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:10.244114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b1ea6-033e-4099-832f-372d17255311 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:10.280075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955afc-4750-4fa6-a659-bbc71aacf353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:10.280872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955afc-4750-4fa6-a659-bbc71aacf353 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:10.281183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955afc-4750-4fa6-a659-bbc71aacf353 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:10.281497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955afc-4750-4fa6-a659-bbc71aacf353 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:10.292712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69955afc-4750-4fa6-a659-bbc71aacf353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-69955afc-4750-4fa6-a659-bbc71aacf353 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:10.311652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:10.312603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:10.313881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 252/487] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:10.326674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955afc-4750-4fa6-a659-bbc71aacf353 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:10.328510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:10.345016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a819f97-b0e1-4368-80fe-110c1f6897ae None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:10.405164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:10.405885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:10.406171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:10.406449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:10.428670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:10.430196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:10.469479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:10.473452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:10.474294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:10.475136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:10.475136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:10.475136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:10.475136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:10.477398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955afc-4750-4fa6-a659-bbc71aacf353 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:10.481566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 236/488] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:10.486555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:10.518863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:10.519537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:10.519822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:10.520101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:10.520854 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:10.521207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:10.522579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:10.539410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 253/489] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:10.544250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:10.672636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e5999-738f-48be-b620-e88d51609105 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:10.708997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bd30a-7c58-4fac-ba5d-08db8d918b25 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:10.735500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:10.736306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:10.736589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:10.736866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:10.754230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:10.756971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:10.758466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:10.775488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:10.776152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:10.776433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:10.776721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:10.777467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:10.777864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:10.779303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:10.783154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:10.783974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:10.785362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:10.785362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:10.785362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:10.785362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:10.785362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 254/490] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:10.790084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:10.805462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:10.813328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 237/491] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:10.817484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:10.925213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00372b04-ef5f-4087-9951-5184a1ac7e4a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:10.955639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f64011-e1d5-48ef-8f5e-67b79c6eeb7d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:10.961372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2faf01-ba23-476c-959f-315b7b9b02fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:10.962037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2faf01-ba23-476c-959f-315b7b9b02fb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:10.962320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2faf01-ba23-476c-959f-315b7b9b02fb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:10.962600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2faf01-ba23-476c-959f-315b7b9b02fb None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:10.963319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2faf01-ba23-476c-959f-315b7b9b02fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:10.963623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2faf01-ba23-476c-959f-315b7b9b02fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:10.964982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2faf01-ba23-476c-959f-315b7b9b02fb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:10.983582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 255/492] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:10.983582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052c84-3191-4c84-8bca-3bed39ed1da2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:10.983582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052c84-3191-4c84-8bca-3bed39ed1da2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:10.983582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed052c84-3191-4c84-8bca-3bed39ed1da2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:10.983582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 256/493] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:10.985014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed052c84-3191-4c84-8bca-3bed39ed1da2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.001713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c678000-1d21-4ab8-8897-3e13da45f90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.002887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c678000-1d21-4ab8-8897-3e13da45f90a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.003372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c678000-1d21-4ab8-8897-3e13da45f90a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.003926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c678000-1d21-4ab8-8897-3e13da45f90a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.012656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c678000-1d21-4ab8-8897-3e13da45f90a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:11.014641 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c678000-1d21-4ab8-8897-3e13da45f90a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0c678000-1d21-4ab8-8897-3e13da45f90a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:11.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:11.034239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:11.035350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:11.037608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 238/494] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:17:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:11.039021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773abc42-3418-47fe-9ac2-c2245ca09b74 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.039434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773abc42-3418-47fe-9ac2-c2245ca09b74 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.039837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773abc42-3418-47fe-9ac2-c2245ca09b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.054938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:11.058040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-773abc42-3418-47fe-9ac2-c2245ca09b74 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:11.059240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-773abc42-3418-47fe-9ac2-c2245ca09b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:11.094424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:11.096557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed052c84-3191-4c84-8bca-3bed39ed1da2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.133291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 239/495] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:11.134797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b583c9-d626-43a8-8104-2262c4a86bc7 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.135157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b583c9-d626-43a8-8104-2262c4a86bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.135504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b583c9-d626-43a8-8104-2262c4a86bc7 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.136838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 240/496] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:11.141258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b583c9-d626-43a8-8104-2262c4a86bc7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.144071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7c8b1-6003-4f96-ab56-080e2e7579c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.145077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c8b1-6003-4f96-ab56-080e2e7579c4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.145564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c8b1-6003-4f96-ab56-080e2e7579c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.146016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7c8b1-6003-4f96-ab56-080e2e7579c4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.154342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7c8b1-6003-4f96-ab56-080e2e7579c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:11.156246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7c8b1-6003-4f96-ab56-080e2e7579c4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-36d7c8b1-6003-4f96-ab56-080e2e7579c4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:11.173375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:11.174252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 03:17:11.175113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:11.177310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 257/497] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:11.178756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3865-dd4f-476d-9f25-d378fa7604be None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.179045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3865-dd4f-476d-9f25-d378fa7604be None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.179322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3865-dd4f-476d-9f25-d378fa7604be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.197672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b6cb3865-dd4f-476d-9f25-d378fa7604be None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:11.198913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b6cb3865-dd4f-476d-9f25-d378fa7604be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:11.244293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b583c9-d626-43a8-8104-2262c4a86bc7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.275117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 258/498] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:11.276276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb3865-dd4f-476d-9f25-d378fa7604be None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.282125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.282793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.283072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.283346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.284172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:11.284485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:11.285977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:11.328775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 241/499] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:17:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:11.328775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.347538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:11.357945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:11.396424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb3865-dd4f-476d-9f25-d378fa7604be None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.428373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1223b-87fe-4886-bfb0-588ddba34c4f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.431478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7865a02-5933-4086-9395-7c94c7828938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.432264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865a02-5933-4086-9395-7c94c7828938 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.432545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865a02-5933-4086-9395-7c94c7828938 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.432820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7865a02-5933-4086-9395-7c94c7828938 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.444534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7865a02-5933-4086-9395-7c94c7828938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c7865a02-5933-4086-9395-7c94c7828938 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:11.452431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:11.453392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:11.454181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 259/500] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:11.454847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7865a02-5933-4086-9395-7c94c7828938 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.466078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.466777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.467061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.467341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.468174 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:11.468485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:11.469967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:11.487491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 242/501] 10.222.0.41 () {56 vars in 1137 bytes} [Sat May 28 03:17:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:11.488269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.553191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7865a02-5933-4086-9395-7c94c7828938 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.566720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:11.572986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:11.584627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a92048-9520-4d41-bfc6-4e27d8a1ccf0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.585651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c22939-7483-430e-9779-bdcec00149d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.586436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22939-7483-430e-9779-bdcec00149d4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.586721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22939-7483-430e-9779-bdcec00149d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.586995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22939-7483-430e-9779-bdcec00149d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.597817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c22939-7483-430e-9779-bdcec00149d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b9c22939-7483-430e-9779-bdcec00149d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:11.605780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:11.606683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:11.607954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 260/502] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:11.609073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304e96-130c-43f7-932f-b420e2b80f86 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.609366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304e96-130c-43f7-932f-b420e2b80f86 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.609677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed304e96-130c-43f7-932f-b420e2b80f86 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.611243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 261/503] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:11.611934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed304e96-130c-43f7-932f-b420e2b80f86 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.624163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecbb17-bf7c-41d4-a274-7249c085c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.624807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecbb17-bf7c-41d4-a274-7249c085c064 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.625088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecbb17-bf7c-41d4-a274-7249c085c064 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.625397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecbb17-bf7c-41d4-a274-7249c085c064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.690794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecbb17-bf7c-41d4-a274-7249c085c064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:11.692317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecbb17-bf7c-41d4-a274-7249c085c064 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:11.712202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed304e96-130c-43f7-932f-b420e2b80f86 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.746236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.746898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.747188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.747502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.748292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:11.748607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:11.750022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:11.758009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 262/504] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:11.758789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.780738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:11.792698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 243/505] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:11.793374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecbb17-bf7c-41d4-a274-7249c085c064 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:11.798496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:11.861248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb17273-0761-4bc0-995c-c796a0ff24c2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.890477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecbb17-bf7c-41d4-a274-7249c085c064 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:11.893026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb774276-03ca-4d69-a3c2-d1364d75acdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.893696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774276-03ca-4d69-a3c2-d1364d75acdb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.893983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774276-03ca-4d69-a3c2-d1364d75acdb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.894264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb774276-03ca-4d69-a3c2-d1364d75acdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.926176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d96a2b-a2e2-4114-bd0b-8a0e1fde1c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:11.926921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d96a2b-a2e2-4114-bd0b-8a0e1fde1c2c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.927209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d96a2b-a2e2-4114-bd0b-8a0e1fde1c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.927487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d96a2b-a2e2-4114-bd0b-8a0e1fde1c2c None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.928226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d96a2b-a2e2-4114-bd0b-8a0e1fde1c2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:11.928540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d96a2b-a2e2-4114-bd0b-8a0e1fde1c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:11.929946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d96a2b-a2e2-4114-bd0b-8a0e1fde1c2c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:11.938333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 244/506] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:11.939496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbae4c-62e1-4d8b-9984-667d3b7e482c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:11.939789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbae4c-62e1-4d8b-9984-667d3b7e482c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:11.945035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbae4c-62e1-4d8b-9984-667d3b7e482c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:11.956732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f8cbae4c-62e1-4d8b-9984-667d3b7e482c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:11.958032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f8cbae4c-62e1-4d8b-9984-667d3b7e482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:11.965634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb774276-03ca-4d69-a3c2-d1364d75acdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:11.967598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb774276-03ca-4d69-a3c2-d1364d75acdb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.000139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.003669 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.023031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 245/507] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:12.023829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbae4c-62e1-4d8b-9984-667d3b7e482c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.067555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 263/508] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:12.068388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb774276-03ca-4d69-a3c2-d1364d75acdb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.120704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbae4c-62e1-4d8b-9984-667d3b7e482c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.155052 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.155834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.156118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.156389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.164755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb774276-03ca-4d69-a3c2-d1364d75acdb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.165940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.167386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:12.185199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:12.186049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:12.186857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:12.186857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:12.186857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.186857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:12.186857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 246/509] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:12.187176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.203272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.204084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.204367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.204646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.207290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.215551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.217572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.221766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:12.235233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:12.236236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:12.237405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:12.237405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:12.237405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.237405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:12.237405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 264/510] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:12.237405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.286580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad3268-705b-4105-af48-ebbf0042537a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.319353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.320147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.320430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.320707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.331508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.332975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.338462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99c697-5b5b-4a43-8bdf-be5810b9cc18 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:12.351631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:12.352483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:12.353433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:12.353433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:12.353433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.353433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:12.353433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 247/511] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:12.353712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.378146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.378938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.379223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.379501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.390187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.391625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:12.409413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:12.410452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:12.411893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:12.411893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:12.411893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.411893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:12.411893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 265/512] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:12.411893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.438440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.451786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.455192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461b9e2-1fc6-459a-aea8-5e4625a22315 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.492886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.493701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.493982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.494260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.504843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.506332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.512503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83807f86-3260-437a-957c-e1563f3e8068 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:12.524153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:12.525194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:12.526238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:12.526238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:12.526238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:12.526238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:12.526238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 248/513] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:12.526238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.549907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.550567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.550853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.551128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.551839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:12.552146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.553552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.565251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 266/514] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:12.566038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.622193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6941c14-1b42-4080-a8b8-d85b7830383c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.664448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.665446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.665870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.666285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.666969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.667396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:12.667933 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.670088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19584ebb-b907-4b87-ae18-107d92787fd5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.670401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.674256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.685653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 249/515] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:12.686743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.705950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.706558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.706830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.707111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.707827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:12.708146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.709553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.722737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 267/516] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:12.724154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.785532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeed42e-6d6c-4be6-81d4-a0b7d8aa2d8a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.818621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.819272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.819683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.819965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.820792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:12.821138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.822513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687a396-0686-41f6-a603-63864d88fd90 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.822616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.834733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 250/517] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:12.835348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.857736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.858393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.858682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.858965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.859682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:12.859989 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.861391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.872396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 268/518] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:12.873269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:12.889863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.895466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:12.932413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbfbbc-9647-4cdb-8ea9-f3b220a603a3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.964989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:12.965741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:12.966020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:12.966854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:12.974406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:12.975853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:12.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d8aa3-7b62-43a8-9355-492d7aff3e9e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:12.989810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:12.990694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 251/519] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:12.996445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.011387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.012180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.012474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.012757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.020154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.021649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:13.036898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:13.037786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:13.038748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:13.038748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:13.038748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:13.038748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:13.038748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:13.038748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:13.038748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:13.038748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:13.039337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 269/520] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:13.040014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.094543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3c2b2-09ba-4139-abd4-e9bbd73cc6d4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.110370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:13.118342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:13.127105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.127888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.128162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.128458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.136588 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.137022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097bf386-4743-49ed-90df-21de60c8881f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.137141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:13.151579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:13.152482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 252/521] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:13.154261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.175749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a79c0-e2a5-482b-a70b-b55e1fa1dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.176413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a79c0-e2a5-482b-a70b-b55e1fa1dd0e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.176693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a79c0-e2a5-482b-a70b-b55e1fa1dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.176965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a79c0-e2a5-482b-a70b-b55e1fa1dd0e None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.177685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a79c0-e2a5-482b-a70b-b55e1fa1dd0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.179101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a79c0-e2a5-482b-a70b-b55e1fa1dd0e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.253548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 270/522] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:17:13 2022] POST /identity/v3/users => generated 247 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:13.254338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a79c0-e2a5-482b-a70b-b55e1fa1dd0e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.255204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041521a9-f4b9-4304-b86b-b12b57307fb4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.293843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.294658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.294944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.295223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.303000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.304646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:13.321790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:13.322915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:13.323949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 253/523] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:13.324518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.367099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:13.396047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a79c0-e2a5-482b-a70b-b55e1fa1dd0e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.432734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.433447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.433727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.434006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.435375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:13.435708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.437176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.440495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2f9e3-ba6b-4c0e-b9a7-6ce41a7bb4f2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.450189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:13.476476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 271/524] 10.222.0.41 () {56 vars in 1137 bytes} [Sat May 28 03:17:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:13.478037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.482039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.482718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.483006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.483286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.484073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:13.484745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.486417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.505715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 254/525] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:13.506592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.587371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c6d35-8cee-4658-a0b5-25d0bbff89a1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.610107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:13.615881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e5717-27a3-42c4-b14a-99ce4eec0f1b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.626119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.627030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.627228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.627516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.628363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:13.628675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.630367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.652503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 272/526] 10.222.0.41 () {56 vars in 1139 bytes} [Sat May 28 03:17:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:13.652503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.653740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.654406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.654688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.654960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.655738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:13.656058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.657538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.670429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 255/527] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:13.675656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.707560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:13.760229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268d137-d071-4a3b-9079-1414bd662652 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.782818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbf8d1-9b9d-4c81-91d8-8f6086380246 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:13.798731 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72f049-d519-48e5-88ee-c72241e95c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.799656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72f049-d519-48e5-88ee-c72241e95c82 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.799943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72f049-d519-48e5-88ee-c72241e95c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.800220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72f049-d519-48e5-88ee-c72241e95c82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.818610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:13.819262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:13.819543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:13.819813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:13.820626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f7ebbd8eb36c4919aedd3498bd8bc483) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:13.820962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.822435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.837143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 256/528] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:13 2022] GET /identity/v3/role_assignments?user.id=f7ebbd8eb36c4919aedd3498bd8bc483&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:13.837143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:13.866880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:13.885094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72f049-d519-48e5-88ee-c72241e95c82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:13.886713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72f049-d519-48e5-88ee-c72241e95c82 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:13.965958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:13.979282 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bbfaf-c474-45c5-9e79-bdb647609ab4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:14.025691 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c8185-15bc-41a0-93f8-e8ed4db4a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:14.026428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c8185-15bc-41a0-93f8-e8ed4db4a38a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:14.026710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c8185-15bc-41a0-93f8-e8ed4db4a38a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:14.026982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c8185-15bc-41a0-93f8-e8ed4db4a38a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:14.028342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 273/529] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:14.029542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72f049-d519-48e5-88ee-c72241e95c82 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:14.112778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c8185-15bc-41a0-93f8-e8ed4db4a38a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:14.114619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c8185-15bc-41a0-93f8-e8ed4db4a38a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:14.137341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:14.154161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72f049-d519-48e5-88ee-c72241e95c82 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:14.192461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:14.193184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:14.193477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:14.193755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:14.194697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=a47589bf9e7c4acba200979e3a91a6ba) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:14.195027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:14.196443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:14.205184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 274/530] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/role_assignments?user.id=a47589bf9e7c4acba200979e3a91a6ba&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:14.205550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:14.226156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:14.250802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 257/531] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:14.257083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c8185-15bc-41a0-93f8-e8ed4db4a38a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:14.344029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99506475-b879-4e35-9c42-64e93aeed484 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:14.368252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c8185-15bc-41a0-93f8-e8ed4db4a38a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:14.391880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:14.395230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:14.396729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:14.397522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:14.398021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:14.407679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:14.408510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:14.408792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:14.409615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:14.415059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:14.416198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:14.420752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:14.422596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:14.440342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:14.441295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:14.442141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:14.442141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:14.442141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:14.442141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:14.443524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:14.444315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:14.445147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:14.445147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:14.445147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:14.445147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:14.445147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 258/532] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:14.446459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 275/533] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:14.452649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:14.452808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:14.542574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:14.566838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a28f5-4817-46f6-b21a-846ce68ad292 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:14.582219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51bfd4-c857-466b-b142-c015be06d4b8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:14.603664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:14.604261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:14.604550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:14.604833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:14.605582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:14.605894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:14.607381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:14.626333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 259/534] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:14.628069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:14.628478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:14.629313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:14.629696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:14.630070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:14.630906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:14.631314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:14.633659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:14.653483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 276/535] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:14.663839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:14.698109 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:14.776594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfbd13-bbd0-4faa-a971-65b4328f2445 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:14.806280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:14.816247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:14.817044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:14.818206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:14.818391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:14.829023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ef60e-dcf8-4eff-ab3f-f4d3fa823628 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:14.833675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:14.835121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:14.860724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:14.861635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:14.863421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 277/536] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:14.870378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:14.895104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:14.896177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:14.896820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:14.898345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:14.913592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:14.913592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:14.943290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:14.946992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:14.947847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 260/537] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:14.958353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:15.070511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1464054-8ac8-48ec-b2b1-897b3f5b79ed None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:15.071348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:15.089847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3d28-081d-442c-8e1c-5153b92a04f6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:15.115100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:15.116142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:15.116626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:15.117126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:15.118172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:15.118684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:15.120341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:15.145991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:15.146667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:15.146960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:15.147238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:15.148032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:15.148345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:15.149826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:15.163746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 278/538] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:15.169842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:15.177200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 261/539] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:15.180676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:15.261985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:15.362017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c25e1-9677-4217-8e1d-dc4da40ddba0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:15.382351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a83151-6561-445b-8e2d-e84ef8e3987d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:15.428640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23221139-b47e-453c-86a8-0c4c88c4a4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:15.429700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23221139-b47e-453c-86a8-0c4c88c4a4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:15.430691 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23221139-b47e-453c-86a8-0c4c88c4a4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:15.431519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23221139-b47e-453c-86a8-0c4c88c4a4a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:15.433098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23221139-b47e-453c-86a8-0c4c88c4a4a2 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=1dd9a0e75308460080902d1da74074c0) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:15.434124 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23221139-b47e-453c-86a8-0c4c88c4a4a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:15.436303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23221139-b47e-453c-86a8-0c4c88c4a4a2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:15.441681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7264d01-6935-4c72-9ac9-75a7aced7068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:15.442628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264d01-6935-4c72-9ac9-75a7aced7068 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:15.443081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264d01-6935-4c72-9ac9-75a7aced7068 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:15.443561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7264d01-6935-4c72-9ac9-75a7aced7068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:15.444573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7264d01-6935-4c72-9ac9-75a7aced7068 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=7092a719870d4b5eafec8d4c51226b65) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:15.445063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7264d01-6935-4c72-9ac9-75a7aced7068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:15.447049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7264d01-6935-4c72-9ac9-75a7aced7068 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:15.451186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 262/540] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:15 2022] GET /identity/v3/role_assignments?user.id=1dd9a0e75308460080902d1da74074c0&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:15.462408 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 279/541] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:15 2022] GET /identity/v3/role_assignments?user.id=7092a719870d4b5eafec8d4c51226b65&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:15.610162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e125c65-807d-46ff-a9a2-ce01688370b1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:15.610482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e125c65-807d-46ff-a9a2-ce01688370b1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:15.610734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e125c65-807d-46ff-a9a2-ce01688370b1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:15.612439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 263/542] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:15.618253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32368-d8ba-480d-a878-b6f92d433b4f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:15.618539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32368-d8ba-480d-a878-b6f92d433b4f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:15.618829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d32368-d8ba-480d-a878-b6f92d433b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:15.646405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-73d32368-d8ba-480d-a878-b6f92d433b4f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:15.647566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-73d32368-d8ba-480d-a878-b6f92d433b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:15.723036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 280/543] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:15.747634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e125c65-807d-46ff-a9a2-ce01688370b1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:15.773373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:15.865782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e125c65-807d-46ff-a9a2-ce01688370b1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:15.913376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b729d-53e0-47be-b226-3e96f1bd26b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:15.914185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b729d-53e0-47be-b226-3e96f1bd26b0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:15.914467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b729d-53e0-47be-b226-3e96f1bd26b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:15.914743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b729d-53e0-47be-b226-3e96f1bd26b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:15.936186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b729d-53e0-47be-b226-3e96f1bd26b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-2e2b729d-53e0-47be-b226-3e96f1bd26b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:15.944404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:15.945313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:15.946104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 264/544] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:15.951543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d32368-d8ba-480d-a878-b6f92d433b4f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:16.008637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:16.114006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d32368-d8ba-480d-a878-b6f92d433b4f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:16.159657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:16.160244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:16.160526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:16.160798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:16.161565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:16.161869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:16.163231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:16.172216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 281/545] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:16.177729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b729d-53e0-47be-b226-3e96f1bd26b0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:16.278422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:16.300904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b729d-53e0-47be-b226-3e96f1bd26b0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:16.336347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617da8b-8463-444e-8632-971afc2748e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:16.337196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617da8b-8463-444e-8632-971afc2748e1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:16.337487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617da8b-8463-444e-8632-971afc2748e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:16.337753 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617da8b-8463-444e-8632-971afc2748e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:16.358510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617da8b-8463-444e-8632-971afc2748e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:16.360191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617da8b-8463-444e-8632-971afc2748e1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4617da8b-8463-444e-8632-971afc2748e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:16.377899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:16.378760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:16.379603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:16.379603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:16.379603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:16.379603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:16.379603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 265/546] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:16.386193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:16.537504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74a2af-0b1e-4a59-8927-837fdae54c5b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:16.542088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:16.583652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:16.584317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:16.584599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:16.584872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:16.585613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:16.586144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:16.587322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:16.600059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5784c-f1fd-4ecc-9763-f71004ceb1ea None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:16.600413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5784c-f1fd-4ecc-9763-f71004ceb1ea None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:16.600783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5784c-f1fd-4ecc-9763-f71004ceb1ea None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:16.602190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 282/547] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:16.602425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 266/548] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:16.607625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:16.611989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ec5a-a282-4bd6-a4f5-c3081962d46e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:16.612362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ec5a-a282-4bd6-a4f5-c3081962d46e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:16.612582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ec5a-a282-4bd6-a4f5-c3081962d46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:16.658105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c578ec5a-a282-4bd6-a4f5-c3081962d46e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:16.659306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c578ec5a-a282-4bd6-a4f5-c3081962d46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:16.739136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cae0f1-0337-4bc2-8c00-6a1e6361af90 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:16.789583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 267/549] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:17:16.794931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:16.795916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:16.796470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:16.796930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:16.804531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:16.806294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:16.821325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:16.822198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:16.823075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:16.824654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578ec5a-a282-4bd6-a4f5-c3081962d46e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:16.835446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 283/550] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:16.838770 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:16.885499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:16.938081 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:17.004381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb874-78a9-4644-97d6-49809c72351d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:17.016400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578ec5a-a282-4bd6-a4f5-c3081962d46e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:17.078416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:17.079040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.079292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.079572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.080410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:17.080723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:17.082189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:17.094551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850f81a-cdfa-4744-a080-fdb12180b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:17.095362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850f81a-cdfa-4744-a080-fdb12180b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.095647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850f81a-cdfa-4744-a080-fdb12180b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.095923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850f81a-cdfa-4744-a080-fdb12180b6f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.108251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850f81a-cdfa-4744-a080-fdb12180b6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-d850f81a-cdfa-4744-a080-fdb12180b6f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:17.117615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:17.118508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:17.119361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 268/551] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:17.123364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 284/552] 10.222.0.41 () {56 vars in 1135 bytes} [Sat May 28 03:17:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:17.128733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850f81a-cdfa-4744-a080-fdb12180b6f1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:17.132300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:17.263478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:17.286786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db0d5c-d2ba-478a-b052-b478198c0c95 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:17.287609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850f81a-cdfa-4744-a080-fdb12180b6f1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:17.310301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:17.332148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee01274-2eb7-4b70-9ba1-3e6fa7bd317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:17.332767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01274-2eb7-4b70-9ba1-3e6fa7bd317b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.333044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01274-2eb7-4b70-9ba1-3e6fa7bd317b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.333363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01274-2eb7-4b70-9ba1-3e6fa7bd317b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.334461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806107c-48d5-457b-982d-d23c8056000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:17.335339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806107c-48d5-457b-982d-d23c8056000b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.335954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806107c-48d5-457b-982d-d23c8056000b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.336413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806107c-48d5-457b-982d-d23c8056000b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.337961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806107c-48d5-457b-982d-d23c8056000b None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:17.338279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806107c-48d5-457b-982d-d23c8056000b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:17.339670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b806107c-48d5-457b-982d-d23c8056000b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:17.351543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 285/553] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:17.353192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272b1f2-f98a-4acc-af16-0b75a31cdedd None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.353700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272b1f2-f98a-4acc-af16-0b75a31cdedd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.354187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272b1f2-f98a-4acc-af16-0b75a31cdedd None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.360012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 286/554] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:17.361509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272b1f2-f98a-4acc-af16-0b75a31cdedd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:17.429372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee01274-2eb7-4b70-9ba1-3e6fa7bd317b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:17.430870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee01274-2eb7-4b70-9ba1-3e6fa7bd317b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:17.503074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272b1f2-f98a-4acc-af16-0b75a31cdedd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:17.544748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 269/555] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:17.546205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73921a61-706d-4e81-8be2-2d29bbead18e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.546515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73921a61-706d-4e81-8be2-2d29bbead18e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.546811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73921a61-706d-4e81-8be2-2d29bbead18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.548238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:17.549472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.549960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.550486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.564307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-73921a61-706d-4e81-8be2-2d29bbead18e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:17.565505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-73921a61-706d-4e81-8be2-2d29bbead18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:17.569577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:17.571141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:17.574026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:17.583040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:17.592867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:17.594689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:17.595535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:17.595535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:17.595535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:17.595535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:17.599478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 287/556] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:17.600424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:17.660765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 270/557] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:17.660765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73921a61-706d-4e81-8be2-2d29bbead18e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:17.722780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846564b7-c7f4-4cad-949b-134510bd8c9f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:17.758827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:17.759343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.759629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.759908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.772553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:17.774269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:17.792564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:17.794764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:17.795626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:17.795626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:17.795626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:17.795626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:17.798901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 288/558] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:17.799389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73921a61-706d-4e81-8be2-2d29bbead18e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:17.799772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:17.848948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:17.850328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.851103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.851583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.852682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:17.853591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:17.855524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:17.855796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:17.873397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 271/559] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:17.874290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:17.886800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:17.931220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b8fcd-9250-4c72-a975-0bc8b0b8146c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:17.977859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a473cd8-7aba-4af8-ab9a-0f52efd7bb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:17.979187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a473cd8-7aba-4af8-ab9a-0f52efd7bb29 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:17.980097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a473cd8-7aba-4af8-ab9a-0f52efd7bb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:17.980741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a473cd8-7aba-4af8-ab9a-0f52efd7bb29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:17.997058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a473cd8-7aba-4af8-ab9a-0f52efd7bb29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:18.004845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eff82-4db8-40bf-a423-57d82ebe7947 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4a473cd8-7aba-4af8-ab9a-0f52efd7bb29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:18.008521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:18.009844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:18.010656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:18.013237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 289/560] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:18.014224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a473cd8-7aba-4af8-ab9a-0f52efd7bb29 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.045955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c701e-5e2a-446b-96c1-3c86067bfabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.046621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c701e-5e2a-446b-96c1-3c86067bfabe None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.046905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c701e-5e2a-446b-96c1-3c86067bfabe None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.047180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c701e-5e2a-446b-96c1-3c86067bfabe None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.048095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c701e-5e2a-446b-96c1-3c86067bfabe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:18.048529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c701e-5e2a-446b-96c1-3c86067bfabe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.049923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c701e-5e2a-446b-96c1-3c86067bfabe None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.067464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 272/561] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:18.068686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20df88-236b-4b24-8e26-1a8c1b006bd5 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.068985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20df88-236b-4b24-8e26-1a8c1b006bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.069294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20df88-236b-4b24-8e26-1a8c1b006bd5 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.073246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 273/562] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:18.073246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20df88-236b-4b24-8e26-1a8c1b006bd5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.120421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a473cd8-7aba-4af8-ab9a-0f52efd7bb29 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.130164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:18.143890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:18.156437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.157257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.157541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.157817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.165767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.167636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.179264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20df88-236b-4b24-8e26-1a8c1b006bd5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:18.181270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:18.182213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 290/563] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:18.183658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.217275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead95b5d-d0f1-440e-85a4-6cd88c64f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.217937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead95b5d-d0f1-440e-85a4-6cd88c64f6ab None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.218218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead95b5d-d0f1-440e-85a4-6cd88c64f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.218494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead95b5d-d0f1-440e-85a4-6cd88c64f6ab None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.219225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead95b5d-d0f1-440e-85a4-6cd88c64f6ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:18.219527 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead95b5d-d0f1-440e-85a4-6cd88c64f6ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.220926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead95b5d-d0f1-440e-85a4-6cd88c64f6ab None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.229337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 274/564] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:18.230514 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68a789-0fc2-47f6-a6b8-a4a8f76f07b5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.230800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68a789-0fc2-47f6-a6b8-a4a8f76f07b5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.231076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68a789-0fc2-47f6-a6b8-a4a8f76f07b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.246637 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7e68a789-0fc2-47f6-a6b8-a4a8f76f07b5 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:18.247854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7e68a789-0fc2-47f6-a6b8-a4a8f76f07b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:18.281980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0254a0-78a4-486f-90c4-913b967b44a6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.314110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 275/565] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:18.314983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.315529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295da27-fa11-4c7c-aa27-68bee1900fda None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.315762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.315865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295da27-fa11-4c7c-aa27-68bee1900fda None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.316050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.316146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295da27-fa11-4c7c-aa27-68bee1900fda None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.316325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.317493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 276/566] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:18.322141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295da27-fa11-4c7c-aa27-68bee1900fda None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.332829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.334313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:18.349002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:18.349900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:18.350755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 291/567] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:18.351217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.378901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:18.385972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:18.430745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295da27-fa11-4c7c-aa27-68bee1900fda None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.451796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8b146-d9c6-4755-a754-5efe37fbfae5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.463850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b722a1-5c87-4b8c-873b-8550336b68c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.464519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b722a1-5c87-4b8c-873b-8550336b68c0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.464801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b722a1-5c87-4b8c-873b-8550336b68c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.465088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b722a1-5c87-4b8c-873b-8550336b68c0 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.466475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b722a1-5c87-4b8c-873b-8550336b68c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:18.466788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b722a1-5c87-4b8c-873b-8550336b68c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.468243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b722a1-5c87-4b8c-873b-8550336b68c0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.485960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.486317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 277/568] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:18.486747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.487032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.487342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.487502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d953a2-c60d-4f32-ac0d-05a22eff7e19 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.487844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d953a2-c60d-4f32-ac0d-05a22eff7e19 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.488067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d953a2-c60d-4f32-ac0d-05a22eff7e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.499103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.500552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.503162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-57d953a2-c60d-4f32-ac0d-05a22eff7e19 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:18.504336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-57d953a2-c60d-4f32-ac0d-05a22eff7e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:18.517664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:18.518552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:18.519421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:18.519421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:18.519421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:18.519421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:18.521418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 292/569] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:18.522107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.574730 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 278/570] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:18.575589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d953a2-c60d-4f32-ac0d-05a22eff7e19 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.604352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:18.623015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2faa40-9973-4c7f-9afd-9386ff0e679e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.623633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:18.656827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea157a-3ba0-4675-b78b-a5db23494859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.657587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea157a-3ba0-4675-b78b-a5db23494859 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.657875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea157a-3ba0-4675-b78b-a5db23494859 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.658172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea157a-3ba0-4675-b78b-a5db23494859 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.668406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea157a-3ba0-4675-b78b-a5db23494859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:18.673047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d953a2-c60d-4f32-ac0d-05a22eff7e19 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2fea157a-3ba0-4675-b78b-a5db23494859 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:18.676761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:18.677819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:18.678630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 293/571] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:18.679308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea157a-3ba0-4675-b78b-a5db23494859 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.706955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.707627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.707909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.708189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.709023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:18.709366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.710876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.723185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 279/572] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:18.723990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.792196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea157a-3ba0-4675-b78b-a5db23494859 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.821957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cc5f0-c133-4b01-8d25-ad1adbc3c959 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.824262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.825044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.825898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.826179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.831044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:18.834681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:18.836911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.838386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.855347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:18.855802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:18.856838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:18.857855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:18.857855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:18.857855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:18.857855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:18.858072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:18.858072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:18.858072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:18.858072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:18.858329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 294/573] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:18.858403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:18.858478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.859154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:18.871691 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 280/574] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:18.875162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:18.995533 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05653085-711d-4b45-bc27-ced4c3512d46 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:18.996439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7d5fa-df71-42cb-be27-629e66db4d36 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.027930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.028411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296eaba-a4d0-4680-a5f4-61836cc2169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.029395 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.029503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.029589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296eaba-a4d0-4680-a5f4-61836cc2169d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.029769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.029859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296eaba-a4d0-4680-a5f4-61836cc2169d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.030035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296eaba-a4d0-4680-a5f4-61836cc2169d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.030507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:19.030812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.032177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.039882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296eaba-a4d0-4680-a5f4-61836cc2169d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:19.040024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 281/575] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:19.040666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4296eaba-a4d0-4680-a5f4-61836cc2169d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:19.047997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:19.049145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 295/576] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:19.050019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296eaba-a4d0-4680-a5f4-61836cc2169d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.081039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:19.109303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:19.148258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf27cf-e1d5-4429-a6fd-7befb077e769 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.154980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296eaba-a4d0-4680-a5f4-61836cc2169d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.181895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.182550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.182827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.183099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.183915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f2f14380f60f4fa18ed6a9579cfb071b) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:19.184228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.186184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.189223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.189904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.190190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.190467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.191178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:19.191386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 282/577] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/role_assignments?user.id=f2f14380f60f4fa18ed6a9579cfb071b&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:19.191481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.192146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.192830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.216977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 296/578] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:19.216977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.302709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:19.310593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f628eb0-090d-40f7-a181-9c1a5c5ec7d6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.327747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7549f-2cf7-4598-8ac0-df83cbdce531 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.341918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:19.353796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.354965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.355510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.355790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.365407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.365605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.366294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.366577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.366853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.366948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.378910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.380352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:19.383768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:19.384664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 283/579] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:19.385769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:19.401318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:19.402258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:19.403140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:19.403140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:19.403140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:19.403140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:19.403140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 297/580] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:19.403427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.505958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efe6f2-02b4-4e48-86d2-67a6787ec6b8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.509285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2aa40-3ab9-4eab-b5df-a7352b36b80c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.540479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.541169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.541457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.541732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.542468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:19.542779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.543388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.544688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.544788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.544977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.545289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.554440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.555940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.558272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 284/581] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:19.559134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:19.571062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:19.571986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:19.572859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 298/582] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:19.574156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.582737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:19.600623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:19.666629 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad416f8-1204-407b-b844-076f92b3a1c7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.677761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e7b28-7558-4658-a048-98bc24464a49 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.703119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.704104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.704563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.705012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.706056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:19.706509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.708100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.724163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.724846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.725161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.725444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.726185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:19.726501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.727920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.729921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 285/583] 10.222.0.41 () {56 vars in 1137 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:19.732208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.756026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 299/584] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:19.756026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.814869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:19.839148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bd923-8fb2-4d22-b5a3-d7fc40b7d4ae None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.851496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:19.868347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2dc22-d4dc-4eee-92a6-dd4c4913cdfb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:19.878856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.879655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.879940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.880217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.891278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.892732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.909732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:19.910394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:19.910674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:19.910948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:19.911729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:19.912039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:19.913508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:19.915903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:19.916758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:19.917679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:19.917679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:19.917679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:19.917679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:19.917679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 286/585] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:19.917950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:19.936758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 300/586] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:19.936758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.036607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370201e-b863-4c14-a8fa-8d4438de9f5c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.062256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:20.066931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9373af-e0ae-48fa-9abd-f98e8d856b1b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.071255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.072049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.072330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.072621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.082881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.084704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:20.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:20.103973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:20.104847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:20.104847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:20.104847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:20.104847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:20.105064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.105064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.105685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.105685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.109163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 287/587] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:20.110216 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.115092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.116523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:20.131423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:20.132334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 301/588] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:20.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.138213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:20.215814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b833ed-837b-4ba7-a822-a87ef8dc5a25 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.241577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e57cb-78aa-4f5d-bb7a-79836647a009 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.250309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.250968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.251248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.251522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.252229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:20.252533 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.253927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.264755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 288/589] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:20.265560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.283169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cf655-5ad8-40e6-afd3-2dee5b2d348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.284323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cf655-5ad8-40e6-afd3-2dee5b2d348d None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.284831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cf655-5ad8-40e6-afd3-2dee5b2d348d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.285434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cf655-5ad8-40e6-afd3-2dee5b2d348d None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f7ebbd8eb36c4919aedd3498bd8bc483/roles/e648d64a4d3d440ba9feaa50a398c2e6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.303359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cf655-5ad8-40e6-afd3-2dee5b2d348d None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=f7ebbd8eb36c4919aedd3498bd8bc483, role_id=e648d64a4d3d440ba9feaa50a398c2e6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.304836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cf655-5ad8-40e6-afd3-2dee5b2d348d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.307725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:20.366193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 302/590] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:17:20 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f7ebbd8eb36c4919aedd3498bd8bc483/roles/e648d64a4d3d440ba9feaa50a398c2e6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:17:20.367046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cf655-5ad8-40e6-afd3-2dee5b2d348d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.383590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828019c7-f0a5-4da6-986a-c09ee677dac9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.428127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.429407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.429864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.430238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.431352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:20.431685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.433306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.448107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:20.455515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 289/591] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:20.456693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.504510 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cf655-5ad8-40e6-afd3-2dee5b2d348d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.543418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.544494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.544965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.545496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.546546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:20.547050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.548697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.572698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 303/592] 10.222.0.41 () {56 vars in 1139 bytes} [Sat May 28 03:17:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:20.573968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.587817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:20.592820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3ce1e-5a79-46b7-bb71-3c5643a8de34 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.639757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.640849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.641164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.641448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.651843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.654542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:20.670720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:20.671716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 03:17:20.672664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:20.680065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 290/593] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:20.681002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.692778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfbba6a-2135-47b0-a043-f1500876de0e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.711856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:20.746913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.748038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.748475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.748900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.761644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.763370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:20.783818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:20.784793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:20.785899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:20.785899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:20.785899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:20.785899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:20.785899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:20.785899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:20.785899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:20.785899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:20.788035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 304/594] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:20.789020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.818535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d3b9c-2fed-4c77-ab02-2c22f25817a2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.857456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:20.858354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:20.858560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:20.858814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:20.875598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:20.875598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:20.895537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:20.900465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:20.901404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:20.901404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:20.901404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:20.901404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:20.909972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 291/595] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:20.910464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:20.934053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18191923-8e6f-48d9-9782-49bc9c12051b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:20.941498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:21.024358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.025342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.025806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.026247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.028343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:21.028343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.030076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.055470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea1ded-d6fd-419d-b41e-a742b8d1cc19 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.061422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 305/596] 10.222.0.41 () {56 vars in 1137 bytes} [Sat May 28 03:17:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:21.062394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.092729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.093429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.093715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.093996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.094785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:21.095100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.098013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.099724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:21.111453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 292/597] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:21.118474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.183333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cc99c-8892-4d25-b530-ab4cd5296b06 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.188699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:21.224356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8f7a-0e81-4342-9905-dd71da3bb664 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.229331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b05df8-9041-4406-b856-058d280f421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.230007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b05df8-9041-4406-b856-058d280f421a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.230295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b05df8-9041-4406-b856-058d280f421a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.230574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b05df8-9041-4406-b856-058d280f421a None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.231309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b05df8-9041-4406-b856-058d280f421a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:21.231671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b05df8-9041-4406-b856-058d280f421a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.233145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b05df8-9041-4406-b856-058d280f421a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.254671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 306/598] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:21.255719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b05df8-9041-4406-b856-058d280f421a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.265968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.266779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.267070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.267351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.279711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.281412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:21.301459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:21.302412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:21.303295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:21.303295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:21.303295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:21.303295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:21.303295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 293/599] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:21.303998 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.390057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:21.404449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b05df8-9041-4406-b856-058d280f421a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.416351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0baa8a-fb4e-46fb-87f2-60aaf64a0378 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.441128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:21.442559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2cbcd-874a-4ad5-89bb-debe0566a997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.443265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2cbcd-874a-4ad5-89bb-debe0566a997 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.444037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2cbcd-874a-4ad5-89bb-debe0566a997 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.444761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2cbcd-874a-4ad5-89bb-debe0566a997 None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/a47589bf9e7c4acba200979e3a91a6ba/roles/e648d64a4d3d440ba9feaa50a398c2e6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.459188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.460505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.460870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.461256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.467893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2cbcd-874a-4ad5-89bb-debe0566a997 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=a47589bf9e7c4acba200979e3a91a6ba, role_id=e648d64a4d3d440ba9feaa50a398c2e6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.469837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.470112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2cbcd-874a-4ad5-89bb-debe0566a997 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.471427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:21.486510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:21.493652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 294/600] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:21.494546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.529192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 307/601] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:17:21 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/a47589bf9e7c4acba200979e3a91a6ba/roles/e648d64a4d3d440ba9feaa50a398c2e6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:17:21.530681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c7e41-54ff-40e8-8232-c4cda1695517 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.530979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c7e41-54ff-40e8-8232-c4cda1695517 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.533425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c7e41-54ff-40e8-8232-c4cda1695517 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.536338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 308/602] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:21.537637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c7e41-54ff-40e8-8232-c4cda1695517 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.609951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8274faf-c6be-4db3-9189-525abe17b00f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.663521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f196ad-7ab5-44b2-bb5c-7b89bfcc8ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.664638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f196ad-7ab5-44b2-bb5c-7b89bfcc8ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.665227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f196ad-7ab5-44b2-bb5c-7b89bfcc8ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.666530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f196ad-7ab5-44b2-bb5c-7b89bfcc8ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.667920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f196ad-7ab5-44b2-bb5c-7b89bfcc8ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:21.668467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f196ad-7ab5-44b2-bb5c-7b89bfcc8ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.671415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f196ad-7ab5-44b2-bb5c-7b89bfcc8ed8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.682602 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c7e41-54ff-40e8-8232-c4cda1695517 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.689925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 295/603] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:21.691758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a5fa-bc0b-4bbf-80ad-9a33234c55b0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.692225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a5fa-bc0b-4bbf-80ad-9a33234c55b0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.692675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a5fa-bc0b-4bbf-80ad-9a33234c55b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.716889 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c3d4a5fa-bc0b-4bbf-80ad-9a33234c55b0 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:21.718505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c3d4a5fa-bc0b-4bbf-80ad-9a33234c55b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:21.735197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.736636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.737252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.737749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.739174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:21.739676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.741328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.757942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:21.759355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:21.763205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 309/604] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:21.765091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.798948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 296/605] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:21.799849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a5fa-bc0b-4bbf-80ad-9a33234c55b0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.890014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc412b3-8cf7-4ac8-9fc9-b8af6924994f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.915262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a5fa-bc0b-4bbf-80ad-9a33234c55b0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:21.932296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.933169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.933467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.933770 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.947192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.947192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:21.952987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe91c3-80b8-478f-969e-8d2f2778b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:21.954132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe91c3-80b8-478f-969e-8d2f2778b836 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:21.954617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe91c3-80b8-478f-969e-8d2f2778b836 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:21.955093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe91c3-80b8-478f-969e-8d2f2778b836 None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/1dd9a0e75308460080902d1da74074c0/roles/e648d64a4d3d440ba9feaa50a398c2e6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:21.961865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:21.962799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:21.963671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:21.963671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:21.963671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:21.963671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:21.963671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:21.963671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:21.963671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:21.963671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:21.968810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 310/606] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:21.969716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:21.977285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe91c3-80b8-478f-969e-8d2f2778b836 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=1dd9a0e75308460080902d1da74074c0, role_id=e648d64a4d3d440ba9feaa50a398c2e6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:21.979147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe91c3-80b8-478f-969e-8d2f2778b836 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:22.033863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:22.074274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26986e9-d3c7-4d77-b167-e5df7305c8f2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:22.074479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 297/607] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:17:21 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/1dd9a0e75308460080902d1da74074c0/roles/e648d64a4d3d440ba9feaa50a398c2e6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:17:22.075904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe91c3-80b8-478f-969e-8d2f2778b836 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:22.113386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:22.113589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500f798-2cf6-4537-b9d2-5d7bf374c07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:22.114337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f798-2cf6-4537-b9d2-5d7bf374c07e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:22.114622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f798-2cf6-4537-b9d2-5d7bf374c07e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:22.114902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f798-2cf6-4537-b9d2-5d7bf374c07e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:22.145562 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500f798-2cf6-4537-b9d2-5d7bf374c07e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d500f798-2cf6-4537-b9d2-5d7bf374c07e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:22.158977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:22.164190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:22.165153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 311/608] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:22.166374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500f798-2cf6-4537-b9d2-5d7bf374c07e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:22.220865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe91c3-80b8-478f-969e-8d2f2778b836 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:22.259004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:22.260056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:22.260547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:22.261048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:22.262107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:22.262624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:22.265532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:22.280978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 298/609] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:22.286922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:22.355587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500f798-2cf6-4537-b9d2-5d7bf374c07e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:22.388975 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:22.401505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:22.415129 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:22.416705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:22.417269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:22.417984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:22.420732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:22.421354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:22.423436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:22.425910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2f164-8052-41f9-9f39-b3f4285152eb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:22.435420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 312/610] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:22.441095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:22.484551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fad0ef-da2f-462c-84f2-30647cd7437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:22.485393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fad0ef-da2f-462c-84f2-30647cd7437a None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:22.485680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fad0ef-da2f-462c-84f2-30647cd7437a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:22.486023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fad0ef-da2f-462c-84f2-30647cd7437a None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/7092a719870d4b5eafec8d4c51226b65/roles/e648d64a4d3d440ba9feaa50a398c2e6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:22.518092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fad0ef-da2f-462c-84f2-30647cd7437a None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=7092a719870d4b5eafec8d4c51226b65, role_id=e648d64a4d3d440ba9feaa50a398c2e6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:22.519557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fad0ef-da2f-462c-84f2-30647cd7437a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:22.586398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c605c-e582-421c-98e6-bd4eca6d3cf1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:22.602960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 299/611] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:17:22 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/7092a719870d4b5eafec8d4c51226b65/roles/e648d64a4d3d440ba9feaa50a398c2e6 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:17:22.633929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:22.634801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:22.635086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:22.635429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:22.647856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:22.649439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:22.670191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:22.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:22.672016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:22.672016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:22.672016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:22.672016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:22.672016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 313/612] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:22.681663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fad0ef-da2f-462c-84f2-30647cd7437a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:22.823433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fad0ef-da2f-462c-84f2-30647cd7437a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:22.868160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:22.871345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98eff6f-1968-4c93-ba50-9b4b2c5248fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:22.872018 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eff6f-1968-4c93-ba50-9b4b2c5248fd None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:22.872337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eff6f-1968-4c93-ba50-9b4b2c5248fd None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:22.872613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98eff6f-1968-4c93-ba50-9b4b2c5248fd None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:22.873360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98eff6f-1968-4c93-ba50-9b4b2c5248fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:22.873665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98eff6f-1968-4c93-ba50-9b4b2c5248fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:22.875055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98eff6f-1968-4c93-ba50-9b4b2c5248fd None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:22.891629 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 300/613] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:22.897066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:22.952881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:23.038247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e790748-a0f7-435c-b09b-c8d86e29a1c8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:23.060431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128869-c0e7-46d3-9565-d066ebc06ef2 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:23.060773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128869-c0e7-46d3-9565-d066ebc06ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:23.060989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128869-c0e7-46d3-9565-d066ebc06ef2 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:23.062381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 301/614] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:23.077890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498bc67-9919-488b-a102-9b6bdd6699ab None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:23.078491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498bc67-9919-488b-a102-9b6bdd6699ab None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:23.078491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498bc67-9919-488b-a102-9b6bdd6699ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:23.096414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d498bc67-9919-488b-a102-9b6bdd6699ab None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:23.098250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d498bc67-9919-488b-a102-9b6bdd6699ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:23.105053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:23.105881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:23.106163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:23.106459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:23.118222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:23.119669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:23.142005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:23.143064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:23.144041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:23.145266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 314/615] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:23.150217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:23.177134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 302/616] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 28 03:17:23.236243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498bc67-9919-488b-a102-9b6bdd6699ab None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:23.285960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d6042-2a28-4e47-bec3-40762c1e4ede None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:23.319558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:23.335645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:23.335645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:23.335645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:23.335645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:23.336052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:23.336052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:23.336052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:23.358315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498bc67-9919-488b-a102-9b6bdd6699ab None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:23.370177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 315/617] 10.222.0.41 () {56 vars in 1135 bytes} [Sat May 28 03:17:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:23.374114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:23.397453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:23.399406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871afaa-5846-4f9d-a1c1-b0a10fcab660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:23.400316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871afaa-5846-4f9d-a1c1-b0a10fcab660 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:23.400703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871afaa-5846-4f9d-a1c1-b0a10fcab660 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:23.401084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871afaa-5846-4f9d-a1c1-b0a10fcab660 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:23.412356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871afaa-5846-4f9d-a1c1-b0a10fcab660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a871afaa-5846-4f9d-a1c1-b0a10fcab660 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:23.426338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:23.427483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:23.428377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 303/618] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:23.433877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871afaa-5846-4f9d-a1c1-b0a10fcab660 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:23.540664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871afaa-5846-4f9d-a1c1-b0a10fcab660 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:23.588942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:23.589949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:23.590432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:23.590939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:23.592648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:23.597531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:23.599151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:23.614026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 304/619] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:23.619406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:23.646395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48978795-9eb2-4c3e-aae1-e7446f7f7542 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:23.707314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:23.710849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:23.711733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:23.711948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:23.712200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:23.737080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:23.738569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:23.772274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:23.773270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:23.774165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:23.774165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:23.774165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:23.774165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:23.783611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 316/620] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:23.784469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:23.793125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:23.814921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8ce5-d249-4afc-afc7-ee8c71ad0851 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:23.869048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:23.870373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:23.870874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:23.871363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:23.892351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:23.895240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:23.926512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:23.927440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:23.928315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:23.928315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:23.928315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:23.928315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:23.942015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 305/621] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:23.949062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:23.969172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842bbaa-88ef-442f-a141-3d777e810ec7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.011029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.011706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.011994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.012278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.013593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:24.013910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:24.015282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:24.045767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 317/622] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:24.053224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:24.134003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:24.158555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:24.169754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654b3df-6c35-44ad-b4ed-ddba2e6b0cd2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.190196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84770e02-e874-40fe-9af0-af0d71690dce None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.208640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0e9ac-9eda-4b6b-a42b-4a6b76dec5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.209445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e9ac-9eda-4b6b-a42b-4a6b76dec5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.209743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e9ac-9eda-4b6b-a42b-4a6b76dec5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.210026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0e9ac-9eda-4b6b-a42b-4a6b76dec5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.210751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0e9ac-9eda-4b6b-a42b-4a6b76dec5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:24.211163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0e9ac-9eda-4b6b-a42b-4a6b76dec5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:24.212922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0e9ac-9eda-4b6b-a42b-4a6b76dec5e2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:24.227105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 306/623] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:24.228555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322fb85-91bf-42ca-b21e-8559dc9b3c3a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.228857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322fb85-91bf-42ca-b21e-8559dc9b3c3a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.229165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322fb85-91bf-42ca-b21e-8559dc9b3c3a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.231405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83232f-72b9-4957-81d3-460974219a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.232521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83232f-72b9-4957-81d3-460974219a8d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.232807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83232f-72b9-4957-81d3-460974219a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.233088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83232f-72b9-4957-81d3-460974219a8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.235158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 307/624] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:24.236076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322fb85-91bf-42ca-b21e-8559dc9b3c3a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:24.241840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83232f-72b9-4957-81d3-460974219a8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:24.243358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83232f-72b9-4957-81d3-460974219a8d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6f83232f-72b9-4957-81d3-460974219a8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:24.261947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:24.262922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:24.263778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:24.263778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:24.263778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:24.263778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:24.263778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:24.263778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:24.263778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:24.263778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:24.264205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 318/625] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:24.266233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490714e-a035-40c3-b16a-bfc101e21a46 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.266525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490714e-a035-40c3-b16a-bfc101e21a46 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.266809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490714e-a035-40c3-b16a-bfc101e21a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.291123 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f490714e-a035-40c3-b16a-bfc101e21a46 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:24.292782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f490714e-a035-40c3-b16a-bfc101e21a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:24.351208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322fb85-91bf-42ca-b21e-8559dc9b3c3a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.382820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 319/626] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:24.383684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490714e-a035-40c3-b16a-bfc101e21a46 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:24.400101 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.400878 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.401220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.401506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.418925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:24.420923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:24.436506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:24.438716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 308/627] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:17:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:24.441034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:24.458255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:24.464249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:24.509788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490714e-a035-40c3-b16a-bfc101e21a46 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.551242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.551923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.552206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.552486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.553455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:24.553708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:24.555263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:24.564008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7acb7-baf6-4416-a8b3-843ab89270a1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.572820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 320/628] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:24.573850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:24.603125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc17b7f-4762-4dad-aca5-d123a997b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.604289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc17b7f-4762-4dad-aca5-d123a997b326 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.604750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc17b7f-4762-4dad-aca5-d123a997b326 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.605218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc17b7f-4762-4dad-aca5-d123a997b326 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.617850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc17b7f-4762-4dad-aca5-d123a997b326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-edc17b7f-4762-4dad-aca5-d123a997b326 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:24.626790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:24.627953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:24.628960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:24.631525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 309/629] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:24.632430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc17b7f-4762-4dad-aca5-d123a997b326 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:24.694554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9c55c-3b0c-4d18-bf71-88b5f1e0f8be None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.708822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:24.733533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.733947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.734303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.734645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.735623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:24.735853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:24.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:24.737303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:24.759881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 321/630] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:17:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:24.759881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:24.764366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc17b7f-4762-4dad-aca5-d123a997b326 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.803208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979caae-bb34-4860-b1df-33efd933f4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.803799 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979caae-bb34-4860-b1df-33efd933f4cf None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.804083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979caae-bb34-4860-b1df-33efd933f4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.804357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979caae-bb34-4860-b1df-33efd933f4cf None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f2f14380f60f4fa18ed6a9579cfb071b/roles/e648d64a4d3d440ba9feaa50a398c2e6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.819110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979caae-bb34-4860-b1df-33efd933f4cf None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=f2f14380f60f4fa18ed6a9579cfb071b, role_id=e648d64a4d3d440ba9feaa50a398c2e6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:24.820521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979caae-bb34-4860-b1df-33efd933f4cf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:24.865561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fe0a0-fbb9-494e-bc6c-f5f227c988d0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:24.891915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 310/631] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:17:24 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f2f14380f60f4fa18ed6a9579cfb071b/roles/e648d64a4d3d440ba9feaa50a398c2e6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:17:24.893707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9a53-df76-442e-9e97-41cd19d26176 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.894238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9a53-df76-442e-9e97-41cd19d26176 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.894758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a9a53-df76-442e-9e97-41cd19d26176 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.896994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 311/632] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:24.898185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a9a53-df76-442e-9e97-41cd19d26176 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:24.917960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ae11c-a4d2-4177-a784-fa582b71f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:24.919052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ae11c-a4d2-4177-a784-fa582b71f6ed None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.920256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ae11c-a4d2-4177-a784-fa582b71f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.926437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ae11c-a4d2-4177-a784-fa582b71f6ed None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.926437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ae11c-a4d2-4177-a784-fa582b71f6ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:24.926437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ae11c-a4d2-4177-a784-fa582b71f6ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:24.926437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ae11c-a4d2-4177-a784-fa582b71f6ed None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:24.942184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 322/633] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:24.944175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47804-27d4-44d7-a2df-271d057e0be2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:24.945380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47804-27d4-44d7-a2df-271d057e0be2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:24.945887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb47804-27d4-44d7-a2df-271d057e0be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:24.971128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ceb47804-27d4-44d7-a2df-271d057e0be2 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:24.972781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ceb47804-27d4-44d7-a2df-271d057e0be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:25.004702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:25.068102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 323/634] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:25.068102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb47804-27d4-44d7-a2df-271d057e0be2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.070849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a9a53-df76-442e-9e97-41cd19d26176 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.112296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:25.122303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63ac02-fe0c-47bf-a2f8-9eadc10bf2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.124912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ac02-fe0c-47bf-a2f8-9eadc10bf2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.124912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ac02-fe0c-47bf-a2f8-9eadc10bf2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.124912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63ac02-fe0c-47bf-a2f8-9eadc10bf2f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.186837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb47804-27d4-44d7-a2df-271d057e0be2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.211472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63ac02-fe0c-47bf-a2f8-9eadc10bf2f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:25.213046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63ac02-fe0c-47bf-a2f8-9eadc10bf2f5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:25.224520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.225345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.225634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.225915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.236924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:25.238435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:25.257385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:25.258277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:25.259155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:25.259155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:25.259155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:25.259155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:25.259155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 324/635] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:25.259875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.278811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:25.322326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 312/636] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:25.323687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a42f29-870c-4ded-9203-50bb954120e5 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.323973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a42f29-870c-4ded-9203-50bb954120e5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.324245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a42f29-870c-4ded-9203-50bb954120e5 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.329146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 313/637] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:25.329146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a42f29-870c-4ded-9203-50bb954120e5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.355019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:25.376302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d5185-aa3b-49cf-bafd-26d0dd79bd96 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.410509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57798a5-df41-4d74-8434-cb8a594132e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.411302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57798a5-df41-4d74-8434-cb8a594132e1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.411582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57798a5-df41-4d74-8434-cb8a594132e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.412047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57798a5-df41-4d74-8434-cb8a594132e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.422579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57798a5-df41-4d74-8434-cb8a594132e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a57798a5-df41-4d74-8434-cb8a594132e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:25.430405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:25.431559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:25.432587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:25.436916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 325/638] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:25.437772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57798a5-df41-4d74-8434-cb8a594132e1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.444609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a42f29-870c-4ded-9203-50bb954120e5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.481990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106667c-929b-41ab-9d59-8259160c112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.482770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106667c-929b-41ab-9d59-8259160c112c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.482898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106667c-929b-41ab-9d59-8259160c112c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.483104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106667c-929b-41ab-9d59-8259160c112c None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.483810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106667c-929b-41ab-9d59-8259160c112c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:25.484117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106667c-929b-41ab-9d59-8259160c112c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:25.485510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106667c-929b-41ab-9d59-8259160c112c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:25.502506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 314/639] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:25.502506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f50f0-a7da-485c-b896-aaa9c5fa7e79 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.502506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f50f0-a7da-485c-b896-aaa9c5fa7e79 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.502506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f50f0-a7da-485c-b896-aaa9c5fa7e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.518518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-673f50f0-a7da-485c-b896-aaa9c5fa7e79 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:25.519699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-673f50f0-a7da-485c-b896-aaa9c5fa7e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:25.524608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:25.546913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57798a5-df41-4d74-8434-cb8a594132e1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.584308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.585192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.585393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.585669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.597222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 315/640] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:25.597222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f50f0-a7da-485c-b896-aaa9c5fa7e79 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.602915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:25.604374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:25.619105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:25.627327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:25.631233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:25.632323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:25.632323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:25.632323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:25.632323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:25.632323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 326/641] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:25.632323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.708722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f50f0-a7da-485c-b896-aaa9c5fa7e79 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.746209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.746791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.747100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.747215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798cc06-8e80-4068-a7eb-9b5306ef21dd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.747382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.748106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:25.748409 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:25.749873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:25.758268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 316/642] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:25.762454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.776807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:25.787217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.788016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.788299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.788581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.797442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:25.801474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:25.812904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:25.814098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:25.815083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 327/643] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:25.815674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.856145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:25.876633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade75ff5-6f60-421b-8cbc-6981fcc27176 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.916638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730ba5c-b532-458a-82c7-ecc0194c8920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.917481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730ba5c-b532-458a-82c7-ecc0194c8920 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.917763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730ba5c-b532-458a-82c7-ecc0194c8920 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.918040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730ba5c-b532-458a-82c7-ecc0194c8920 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.924244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee246e-53bc-4cbe-ae55-6bd8d30596cd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:25.929276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730ba5c-b532-458a-82c7-ecc0194c8920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1730ba5c-b532-458a-82c7-ecc0194c8920 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:25.939787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:25.940716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:25.942011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 317/644] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:25.942791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730ba5c-b532-458a-82c7-ecc0194c8920 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:25.963463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:25.964406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:25.964690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:25.964963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:25.965716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:25.966048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:25.967947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:25.985153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 328/645] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:25.985153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.034766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:26.056512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730ba5c-b532-458a-82c7-ecc0194c8920 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.092380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7c615-1fc4-4118-81a3-ff943b1db696 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.092604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.093328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.093613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.093931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.101007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:26.105431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.106866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:26.124504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:26.125584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:26.126454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:26.126454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:26.126454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:26.126454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:26.126454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 318/646] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:26.127262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.134137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.134808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.135090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.135364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.136196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:26.136505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.138515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.160249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 329/647] 10.222.0.41 () {56 vars in 1137 bytes} [Sat May 28 03:17:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:26.160249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.241429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5c307-a1b4-4d99-a265-eeba909fd149 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.260135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8ab62-e9c6-48f7-b1a6-151a18f89fe5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.267944 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:26.274557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.275518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.275518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.275777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.276559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:26.282927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.282927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.291473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 319/648] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:26.291473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.303074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.304142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.304848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.305156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.314337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.316298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:26.331568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:26.332414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:26.333338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 330/649] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:26.334353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.349263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:26.396448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae5484-8e65-456e-8cf2-32c0f04edc4f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.429934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.430590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.430875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.431155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.431860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:26.432164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.433582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.440873 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e417be10-2a66-4df9-b376-b77c0a435d96 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.450746 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 320/650] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:26.450746 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.479614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cca84e-46a6-4f91-bdd0-dafb3eb24e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.480311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cca84e-46a6-4f91-bdd0-dafb3eb24e36 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.480594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cca84e-46a6-4f91-bdd0-dafb3eb24e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.480864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cca84e-46a6-4f91-bdd0-dafb3eb24e36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.518357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:26.554168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cca84e-46a6-4f91-bdd0-dafb3eb24e36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.554338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf32e0d-b725-4b40-9684-b386e6a9a68b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.555549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cca84e-46a6-4f91-bdd0-dafb3eb24e36 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.568939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:26.590493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.592054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.592845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.593868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.605312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.606869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:26.624823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:26.625971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:26.627041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:26.627041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:26.627041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:26.627041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:26.627041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 321/651] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:26.627330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.675113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 331/652] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:26.680247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cca84e-46a6-4f91-bdd0-dafb3eb24e36 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.734697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37297a25-3dae-4f00-b347-94edfd96c98b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.749910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:26.777743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.778451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.778765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.779047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.779839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:26.780272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.781788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.789126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cca84e-46a6-4f91-bdd0-dafb3eb24e36 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.794447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 322/653] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:26.795581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.826780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:26.831597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.832398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.832825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.832955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.840050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.844982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:26.857144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:26.858376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:26.859295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 332/654] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:26.859894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.918231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b59e4a-4e48-4b4f-a25d-cc2f7ced2785 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:26.953655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:26.954468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:26.954757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:26.955039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:26.955783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:26.956095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:26.957577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:26.971210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 323/655] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:26.972444 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:26.990060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5945a7-7ddd-452f-b747-215cac6924d5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.018066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:27.026613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.035334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.035334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.035334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.041636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:27.043551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:27.064370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:27.065334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:27.066241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:27.066241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:27.066241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:27.066241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:27.066241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 333/656] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:27.067026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.067694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:27.085671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a9395d-05bd-4d70-82fc-8b58c98939b5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.126046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.126957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.127338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.127624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.128452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f7ebbd8eb36c4919aedd3498bd8bc483) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:27.128771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:27.130260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:27.145143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 324/657] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:26 2022] GET /identity/v3/role_assignments?user.id=f7ebbd8eb36c4919aedd3498bd8bc483&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:27.145143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.183079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5ff39-36ad-4d85-b178-38aed1f0c792 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.219578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.220265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.220547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.220820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.221705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:27.222280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:27.223753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:27.243281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 334/658] 10.222.0.41 () {56 vars in 1139 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:27.244140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.265488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1924e2-e0fe-4993-ac6c-b757c61ee6f3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.283969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:27.303742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8c78d-2485-47a5-a81e-d7e1dfcbd53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.304653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8c78d-2485-47a5-a81e-d7e1dfcbd53f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.305015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8c78d-2485-47a5-a81e-d7e1dfcbd53f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.305392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8c78d-2485-47a5-a81e-d7e1dfcbd53f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.315042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8c78d-2485-47a5-a81e-d7e1dfcbd53f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:27.317031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8c78d-2485-47a5-a81e-d7e1dfcbd53f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-30b8c78d-2485-47a5-a81e-d7e1dfcbd53f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:27.333219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:27.334380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:27.335523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 325/659] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:27.336522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78399cd-ac52-4dbe-afec-78e84faa1d85 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.336755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78399cd-ac52-4dbe-afec-78e84faa1d85 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.337185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78399cd-ac52-4dbe-afec-78e84faa1d85 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.338715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 326/660] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:27.339427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78399cd-ac52-4dbe-afec-78e84faa1d85 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.353387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:27.362346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecbd05-4550-436e-a1ca-5288b2f9267f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.401974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af8573-9723-4653-9743-beebd1c94e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.402884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8573-9723-4653-9743-beebd1c94e03 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.403847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8573-9723-4653-9743-beebd1c94e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.404277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af8573-9723-4653-9743-beebd1c94e03 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.404996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af8573-9723-4653-9743-beebd1c94e03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:27.405328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af8573-9723-4653-9743-beebd1c94e03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:27.406699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af8573-9723-4653-9743-beebd1c94e03 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:27.418279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 335/661] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:27.420159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1f174-2687-4f61-aaf5-a64c06305209 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.420380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1f174-2687-4f61-aaf5-a64c06305209 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.420655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1f174-2687-4f61-aaf5-a64c06305209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.436714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-69f1f174-2687-4f61-aaf5-a64c06305209 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:27.437912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-69f1f174-2687-4f61-aaf5-a64c06305209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:27.461370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78399cd-ac52-4dbe-afec-78e84faa1d85 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.495155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62ecc3-e530-4f66-ad4a-93cd434d56a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.495810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62ecc3-e530-4f66-ad4a-93cd434d56a0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.496093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62ecc3-e530-4f66-ad4a-93cd434d56a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.496369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62ecc3-e530-4f66-ad4a-93cd434d56a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.517131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 336/662] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:27.517131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1f174-2687-4f61-aaf5-a64c06305209 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.544889 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:27.572486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62ecc3-e530-4f66-ad4a-93cd434d56a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:27.574038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62ecc3-e530-4f66-ad4a-93cd434d56a0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:27.584853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:27.618545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1f174-2687-4f61-aaf5-a64c06305209 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.652246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.652916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.653757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.654144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.654971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:27.655317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:27.656701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:27.680333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 337/663] 10.222.0.41 () {56 vars in 1137 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:27.681094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.686279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 327/664] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:27.686279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62ecc3-e530-4f66-ad4a-93cd434d56a0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.767961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:27.784374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193f3de-a4a1-4895-adb2-82dc366bba85 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.787610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62ecc3-e530-4f66-ad4a-93cd434d56a0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.797696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:27.821233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.822620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.823170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.823711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.824273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf73a9-f1ce-4926-aed3-84e4ec43d958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:27.827491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf73a9-f1ce-4926-aed3-84e4ec43d958 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:27.827491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf73a9-f1ce-4926-aed3-84e4ec43d958 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:27.827491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf73a9-f1ce-4926-aed3-84e4ec43d958 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:27.833922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:27.835497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:27.839946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf73a9-f1ce-4926-aed3-84e4ec43d958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9baf73a9-f1ce-4926-aed3-84e4ec43d958 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:27.848937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:27.849930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:27.850773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:27.853415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 328/665] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:27.854284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:27.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:27.856012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:27.856012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:27.856012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:27.856012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:27.856012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:27.856012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:27.856012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:27.856012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:27.856821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf73a9-f1ce-4926-aed3-84e4ec43d958 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.860344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 338/666] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:27.861191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:27.970276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf73a9-f1ce-4926-aed3-84e4ec43d958 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:27.970779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8937f8c-43a1-4745-84cd-8bb43d4c655c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.007641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.008431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.008711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.008822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b535-8e76-4051-804d-f7d61bafa229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.008997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.009439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b535-8e76-4051-804d-f7d61bafa229 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.009727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b535-8e76-4051-804d-f7d61bafa229 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.010003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207b535-8e76-4051-804d-f7d61bafa229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.020526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.021996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.024849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:28.043088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:28.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:28.044941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:28.044941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:28.044941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:28.044941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:28.048475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:28.049523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 339/667] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:28.050223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.083608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207b535-8e76-4051-804d-f7d61bafa229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.085155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207b535-8e76-4051-804d-f7d61bafa229 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.159144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553677a-7e37-4bf5-b69e-a467127393c6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.189332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 329/668] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:28.190426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b535-8e76-4051-804d-f7d61bafa229 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.202050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.202850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.203567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.203945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.205149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:28.205691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.207148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.219858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 340/669] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:28.219858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.264551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:28.283724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:28.298256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207b535-8e76-4051-804d-f7d61bafa229 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.326797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beff115-51b9-47a7-acd2-7bff6ef8255c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.334366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.335026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.338783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.338783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.338783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:28.338783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.338783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.350766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 330/670] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:28.351572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.368077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.368743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.369030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.369357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.370082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:28.370395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.371763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.385002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 341/671] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:28.385861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.458969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29f878-70a9-4e31-b36e-3630027c4e48 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.487947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:28.489868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b8e1a-70b1-4f5a-b494-10509753fc12 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.496615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.498092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.498881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.498881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.507487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:28.509260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.511021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.529058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.529888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.530176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.530460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.541915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.543394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:28.564534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:28.565759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:28.566725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:28.566725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:28.566725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:28.566725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:28.570866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 342/672] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:28.570866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:28.649521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:28.650493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:28.651381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:28.651381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:28.651381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:28.651381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:28.651381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 331/673] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:28.651664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.686872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476fd8c-f526-49e3-901a-d3fa0005f473 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.724948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.725895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.726398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.726683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.727660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=a47589bf9e7c4acba200979e3a91a6ba) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:28.727987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.729502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.736085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 343/674] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/role_assignments?user.id=a47589bf9e7c4acba200979e3a91a6ba&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:28.736875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.776937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82455cd9-6b11-4826-9c69-c4587f8d9712 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.780592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:28.821963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.822806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.823455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.823455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.831804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.833563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:28.849779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:28.850766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:28.851645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:28.851645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:28.851645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:28.851645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:28.851645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:28.851645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:28.851645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:28.851645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:28.856230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 332/675] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:28.857822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.877395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1cc1e-7237-419a-9652-3042ccba106f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:28.911344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:28.932204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:28.933491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:28.933783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:28.934062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:28.934828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:28.935139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:28.936500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:28.948835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 344/676] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:28.949526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:28.989863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0616d24d-028e-4934-bf7d-e0026634ecca None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.033237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.034272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.034769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.035236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.035962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:29.036271 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.037737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.057246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 333/677] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:29.058145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:29.077448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:29.087893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cb511-2671-40c4-84f3-1f90c5cbbeb5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.134745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.135659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.136826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.137129 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.137980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:29.138300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.139761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.157388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 345/678] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:29.158259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:29.178227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:29.181259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017886f-8696-4c16-b164-4fc4b8963648 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.217232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.218210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.218598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.218978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.228621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.230445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:29.247336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:29.248531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:29.250571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:29.251923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 334/679] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:29.251923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:29.292416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304fc25-fe1f-4467-b559-42e5a4719616 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.341398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.342452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.342741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.343020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.352857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.354990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.360759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f51284-4777-491e-b64a-00bdd158ea1d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:29.374661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:29.376570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 346/680] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:29.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:29.398823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:29.408235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.409202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.426398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.426398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.426398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=1dd9a0e75308460080902d1da74074c0) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:29.426398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.426398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.426766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 335/681] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/role_assignments?user.id=1dd9a0e75308460080902d1da74074c0&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:29.426766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:29.451490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:29.492260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc7fd5-b78d-4cf5-948b-b60b892678c4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.529450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f21d8a-0c77-487c-a237-3eda251969d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.530127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f21d8a-0c77-487c-a237-3eda251969d1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.530410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f21d8a-0c77-487c-a237-3eda251969d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.530683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f21d8a-0c77-487c-a237-3eda251969d1 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.531528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f21d8a-0c77-487c-a237-3eda251969d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:29.531835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f21d8a-0c77-487c-a237-3eda251969d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.533344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f21d8a-0c77-487c-a237-3eda251969d1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.534672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f1058-73b7-4845-8845-c5ae226fcce4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.556925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 347/682] 10.222.0.41 () {56 vars in 1135 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:29.558463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ffb0-3553-429c-bc02-e474ef1fc091 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.558757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ffb0-3553-429c-bc02-e474ef1fc091 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.559044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ffb0-3553-429c-bc02-e474ef1fc091 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.560718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 348/683] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:29.561504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ffb0-3553-429c-bc02-e474ef1fc091 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:29.571847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80341950-6f2c-4d27-bc5f-f30efaea7b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.572539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80341950-6f2c-4d27-bc5f-f30efaea7b47 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.572827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80341950-6f2c-4d27-bc5f-f30efaea7b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.573192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80341950-6f2c-4d27-bc5f-f30efaea7b47 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.573997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80341950-6f2c-4d27-bc5f-f30efaea7b47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:29.574360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80341950-6f2c-4d27-bc5f-f30efaea7b47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.576425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80341950-6f2c-4d27-bc5f-f30efaea7b47 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.594896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 336/684] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:29.596410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731555b0-14f3-4691-9e1c-82c4a1ea7f5d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.596710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731555b0-14f3-4691-9e1c-82c4a1ea7f5d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.597294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731555b0-14f3-4691-9e1c-82c4a1ea7f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.615694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-731555b0-14f3-4691-9e1c-82c4a1ea7f5d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:29.618009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-731555b0-14f3-4691-9e1c-82c4a1ea7f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:29.649789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:29.683444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ffb0-3553-429c-bc02-e474ef1fc091 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.709165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 337/685] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:29.709165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731555b0-14f3-4691-9e1c-82c4a1ea7f5d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:29.721662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:29.748785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87c011-f42d-4093-897c-3d08d40efb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.749489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c011-f42d-4093-897c-3d08d40efb4e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.749779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c011-f42d-4093-897c-3d08d40efb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.750059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87c011-f42d-4093-897c-3d08d40efb4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.819470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731555b0-14f3-4691-9e1c-82c4a1ea7f5d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:29.826517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87c011-f42d-4093-897c-3d08d40efb4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.828181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87c011-f42d-4093-897c-3d08d40efb4e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.856639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:29.857455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:29.857807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:29.858159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:29.859097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=7092a719870d4b5eafec8d4c51226b65) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:29.859480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:29.860978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:29.873256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 338/686] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/role_assignments?user.id=7092a719870d4b5eafec8d4c51226b65&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:29.874285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:29.919046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:29.959324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 349/687] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:29.966696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87c011-f42d-4093-897c-3d08d40efb4e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:30.020604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:30.050884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35370170-5765-4349-9eec-7d8837a4c2d2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:30.095720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489367ce-bf94-4a68-93e1-14dee2ea48c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:30.096753 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489367ce-bf94-4a68-93e1-14dee2ea48c8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:30.097478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489367ce-bf94-4a68-93e1-14dee2ea48c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:30.097900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489367ce-bf94-4a68-93e1-14dee2ea48c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:30.110600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489367ce-bf94-4a68-93e1-14dee2ea48c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-489367ce-bf94-4a68-93e1-14dee2ea48c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:30.119897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:30.120865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:30.122642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:30.127097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 339/688] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:30.130553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489367ce-bf94-4a68-93e1-14dee2ea48c8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:30.195023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87c011-f42d-4093-897c-3d08d40efb4e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:30.248495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:30.249468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:30.249762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:30.250325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:30.263369 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:30.265177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:30.267604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489367ce-bf94-4a68-93e1-14dee2ea48c8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:30.302626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:30.303600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:30.304472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:30.304472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:30.304472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:30.304472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:30.309186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 350/689] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:30.312213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:30.333159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:30.333875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:30.334157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:30.334429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:30.335171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:30.341314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:30.341314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:30.346329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 340/690] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:30.346574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:30.352769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:30.369063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:30.504971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63888e39-1488-43e4-a02f-29dabf3472ca None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:30.547213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:30.548023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:30.548326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:30.548610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:30.549365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:30.549680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:30.551247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:30.562996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19cce0-225a-4158-9262-592492361dbb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:30.564575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 351/691] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:30.570428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:30.631073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:30.632231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:30.632712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:30.633230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:30.652528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:30.654035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:30.674796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:30.675797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:30.676674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:30.676674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:30.676674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:30.676674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:30.676674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 341/692] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:30.681787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:30.737071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:30.772264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a560a5e-dd7c-4d10-b88f-0d6287734b72 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:30.780155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:30.836617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:30.837448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:30.837735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:30.838016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:30.848177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:30.849663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:30.864839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:30.865769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:30.866644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 352/693] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:30.873879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:30.918170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852ebd6-44f0-460f-b5f9-75866a68a485 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:30.987530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:30.988199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:30.988481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:30.988755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:30.989505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:30.989811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:30.991190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:31.020074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 342/694] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:31.025442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:31.030382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ef732-5038-4531-a355-3f49f2906bec None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:31.078905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:31.079563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.079845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.080127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.080905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:31.081240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:31.082653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:31.108707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 353/695] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:31.123426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:31.132046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:31.178168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd874d7-297f-41fc-94ca-5a3bf40c6e8e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:31.201476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:31.218115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:31.219106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.220635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.221311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.233275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:31.234983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:31.251970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:31.253096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:31.255095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:31.255988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 343/696] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:17:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:31.261555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:31.319364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85404b1f-198f-4ef3-858e-c775a77907e0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:31.354870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50260344-ecce-41db-aaba-201bf000e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:31.355534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260344-ecce-41db-aaba-201bf000e020 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.355819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260344-ecce-41db-aaba-201bf000e020 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.356097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50260344-ecce-41db-aaba-201bf000e020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.357694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50260344-ecce-41db-aaba-201bf000e020 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f2f14380f60f4fa18ed6a9579cfb071b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:31.358016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50260344-ecce-41db-aaba-201bf000e020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:31.359452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50260344-ecce-41db-aaba-201bf000e020 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:31.365774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 354/697] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:31 2022] GET /identity/v3/role_assignments?user.id=f2f14380f60f4fa18ed6a9579cfb071b&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:31.367096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc0a7b-a159-4dcc-9d1d-78f1554847da None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.367391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc0a7b-a159-4dcc-9d1d-78f1554847da None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.367674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc0a7b-a159-4dcc-9d1d-78f1554847da None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.369204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 355/698] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:31.380860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1af51-e942-48c6-aff3-52d36b6e0112 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:31.386076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ee33c-6351-4baf-8007-0a5077af8828 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.386356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ee33c-6351-4baf-8007-0a5077af8828 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.386628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ee33c-6351-4baf-8007-0a5077af8828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.406276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-676ee33c-6351-4baf-8007-0a5077af8828 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:31.407725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-676ee33c-6351-4baf-8007-0a5077af8828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:31.458221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:31.458835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.459123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.459400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.460256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:31.460577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:31.462097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:31.496820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 344/699] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:17:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:31.501652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:31.503408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:31.504687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 356/700] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:17:31.537418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:31.556193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ee33c-6351-4baf-8007-0a5077af8828 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:31.658152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f9617-31a1-4fbc-b594-4d7ab9b2ded1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:31.694697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba13d45-b5eb-4cb7-9709-1d656b0ff9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:31.695339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba13d45-b5eb-4cb7-9709-1d656b0ff9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.695619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba13d45-b5eb-4cb7-9709-1d656b0ff9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.695892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba13d45-b5eb-4cb7-9709-1d656b0ff9d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.696324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ee33c-6351-4baf-8007-0a5077af8828 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:31.737096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:31.737913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.738197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.738469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.749560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:31.751201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:31.770280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba13d45-b5eb-4cb7-9709-1d656b0ff9d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:31.770656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:31.772705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:31.774914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:31.774914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:31.774914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:31.774914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:31.775530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba13d45-b5eb-4cb7-9709-1d656b0ff9d5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:31.781126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 357/701] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:31.781126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:31.802024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:31.838680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:31.884344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 345/702] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:31.891132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba13d45-b5eb-4cb7-9709-1d656b0ff9d5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:31.894763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0004a7f-7b0d-4780-83fa-11a8a5994b22 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:31.935705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:31.936640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:31.937087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:31.937539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:31.939242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:31.939733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:31.941340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:31.955293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 358/703] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:31.960518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:31.999050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba13d45-b5eb-4cb7-9709-1d656b0ff9d5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:32.033712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:32.040541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a045ad-a6d4-4a3a-8ec6-d7e8a1cdb1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:32.048431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a045ad-a6d4-4a3a-8ec6-d7e8a1cdb1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.048431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a045ad-a6d4-4a3a-8ec6-d7e8a1cdb1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.048431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a045ad-a6d4-4a3a-8ec6-d7e8a1cdb1d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.054492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a045ad-a6d4-4a3a-8ec6-d7e8a1cdb1d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:32.056254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a045ad-a6d4-4a3a-8ec6-d7e8a1cdb1d2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f2a045ad-a6d4-4a3a-8ec6-d7e8a1cdb1d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:32.078072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:32.079002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:32.079910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:32.079910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:32.079910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:32.079910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:32.085051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 346/704] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:32.086859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025bdf1-1be3-4aae-ae2c-498cdafbdf4c None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.087157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025bdf1-1be3-4aae-ae2c-498cdafbdf4c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.087432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025bdf1-1be3-4aae-ae2c-498cdafbdf4c None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.090163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 347/705] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:32.090278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178e55e-7b77-4413-9f63-835e198a4c0e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:32.090359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025bdf1-1be3-4aae-ae2c-498cdafbdf4c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:32.104272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:32.141858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c85b0-12fe-4e37-8998-553d495f82ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:32.143564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c85b0-12fe-4e37-8998-553d495f82ff None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.144179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c85b0-12fe-4e37-8998-553d495f82ff None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.144661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c85b0-12fe-4e37-8998-553d495f82ff None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.145878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c85b0-12fe-4e37-8998-553d495f82ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:32.148281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c85b0-12fe-4e37-8998-553d495f82ff None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:32.204511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025bdf1-1be3-4aae-ae2c-498cdafbdf4c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:32.236680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 359/706] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:17:31 2022] POST /identity/v3/users => generated 254 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:32.237787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0620ff-e639-4d7c-8048-1579ebb27542 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.238077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0620ff-e639-4d7c-8048-1579ebb27542 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.238374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0620ff-e639-4d7c-8048-1579ebb27542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.258889 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:32.260074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.260245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2a0620ff-e639-4d7c-8048-1579ebb27542 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:32.260754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.261264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.261474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2a0620ff-e639-4d7c-8048-1579ebb27542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:32.262404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:32.263553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:32.266785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:32.284729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 348/707] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:32.291149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:32.383161 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:32.385201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 360/708] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:32.432881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0620ff-e639-4d7c-8048-1579ebb27542 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:32.478562 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f7732-bd23-404a-9298-010ff82af012 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:32.526074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:32.527748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.528274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.528758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.538300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:32.539968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:32.549510 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:32.557885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:32.558768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:32.560205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:32.560205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:32.560205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:32.560205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:32.560205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:32.560205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:32.560205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:32.560205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:32.560661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0620ff-e639-4d7c-8048-1579ebb27542 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:32.560739 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 349/709] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:32.567042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:32.597316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c992-e6c7-42ab-9ada-a51937b320ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:32.598414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c992-e6c7-42ab-9ada-a51937b320ca None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.598847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c992-e6c7-42ab-9ada-a51937b320ca None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.599270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c992-e6c7-42ab-9ada-a51937b320ca None admin] PATH_INFO: `/v3/services/network` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.600096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c992-e6c7-42ab-9ada-a51937b320ca None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:32.602224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c992-e6c7-42ab-9ada-a51937b320ca None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d159c992-e6c7-42ab-9ada-a51937b320ca None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:32.618955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:32.620077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:32.620932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:32.624466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 361/710] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:32 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:32.626023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f733bce-62dc-4969-857d-7243625baf75 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.626541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f733bce-62dc-4969-857d-7243625baf75 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.626993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f733bce-62dc-4969-857d-7243625baf75 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.629185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 362/711] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:32.634584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af172c92-cb97-44f5-a706-5bbef77e50f1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.636226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af172c92-cb97-44f5-a706-5bbef77e50f1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.636507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af172c92-cb97-44f5-a706-5bbef77e50f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.660282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-af172c92-cb97-44f5-a706-5bbef77e50f1 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:32.661502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-af172c92-cb97-44f5-a706-5bbef77e50f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:32.703932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245a5ee-4828-4dae-878f-ab3e10835a21 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:32.706919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:32.738065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 363/712] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:32.740040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af172c92-cb97-44f5-a706-5bbef77e50f1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:32.758302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:32.759310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.759793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.760372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.761506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:32.762022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:32.765144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:32.778686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 350/713] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:32.779847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:32.852230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:32.862430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af172c92-cb97-44f5-a706-5bbef77e50f1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:32.890794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833498b7-e307-47ae-8380-5867c60cd8ed None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:32.898272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:32.899052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.899256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.899496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.900201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] RBAC: Adding query filter params (name=network) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:32.900507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:32.902509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:32.916250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 364/714] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:17:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:32.921424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:32.929202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:32.930693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:32.931203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:32.931679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:32.932528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:32.934144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:32.949338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:32.950861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:32.951687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:32.954564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 351/715] 10.222.0.41 () {56 vars in 1129 bytes} [Sat May 28 03:17:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:32.955655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:32.991216 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:33.051678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b6dbd-8b14-4898-b593-1ccd27e61d27 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:33.079959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d155a-5693-470a-b6f3-36c4820a27d2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:33.087116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:33.087769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:33.088052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:33.088328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:33.089187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f7ebbd8eb36c4919aedd3498bd8bc483) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:33.089505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:33.091566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:33.105442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 365/716] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:32 2022] GET /identity/v3/role_assignments?user.id=f7ebbd8eb36c4919aedd3498bd8bc483&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=bb0756f8af694633b9743a958eb46e7a => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:33.106505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:33.117673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:33.118348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:33.118632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:33.118914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:33.119624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] RBAC: Adding query filter params (type=network) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:33.119932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:33.121328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:33.127855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 352/717] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:17:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:33.140939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:33.247132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3cbdd-43d3-49f2-8e81-fd8f823b5c98 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:33.288718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28bee9-e551-4c64-a241-efcb9af193f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:33.289474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28bee9-e551-4c64-a241-efcb9af193f4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:33.289758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28bee9-e551-4c64-a241-efcb9af193f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:33.290038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28bee9-e551-4c64-a241-efcb9af193f4 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:33.290747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28bee9-e551-4c64-a241-efcb9af193f4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:33.291052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28bee9-e551-4c64-a241-efcb9af193f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:33.292483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28bee9-e551-4c64-a241-efcb9af193f4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:33.298287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 366/718] 10.222.0.41 () {56 vars in 1124 bytes} [Sat May 28 03:17:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:33.303829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:33.444537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:33.454976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7b8dd-42da-496c-a695-8bb3a15d11df None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:33.492642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c0729-58cb-49d2-a0b7-ed3401e0e014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:33.493346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c0729-58cb-49d2-a0b7-ed3401e0e014 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:33.493640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c0729-58cb-49d2-a0b7-ed3401e0e014 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:33.493928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c0729-58cb-49d2-a0b7-ed3401e0e014 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:33.494650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c0729-58cb-49d2-a0b7-ed3401e0e014 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:33.494968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c0729-58cb-49d2-a0b7-ed3401e0e014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:33.498446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c0729-58cb-49d2-a0b7-ed3401e0e014 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:33.503458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 353/719] 10.222.0.41 () {56 vars in 1124 bytes} [Sat May 28 03:17:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:34.442410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693a68e-8f79-48d6-9884-945ccfab4b23 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:34.442948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693a68e-8f79-48d6-9884-945ccfab4b23 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:34.442948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693a68e-8f79-48d6-9884-945ccfab4b23 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:34.444571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 367/720] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:34.450298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5ed04-5406-4e5a-baf6-179978a8ab49 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:34.450711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5ed04-5406-4e5a-baf6-179978a8ab49 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:34.450883 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5ed04-5406-4e5a-baf6-179978a8ab49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:34.474080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-24b5ed04-5406-4e5a-baf6-179978a8ab49 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:34.475264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-24b5ed04-5406-4e5a-baf6-179978a8ab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:34.572615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 354/721] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 03:17:34.608188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693a68e-8f79-48d6-9884-945ccfab4b23 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:34.740800 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:34.771040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693a68e-8f79-48d6-9884-945ccfab4b23 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:34.820687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07dec9-e80b-4fdb-bdb3-f70750fc292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:34.821525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07dec9-e80b-4fdb-bdb3-f70750fc292c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:34.821816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07dec9-e80b-4fdb-bdb3-f70750fc292c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:34.822097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07dec9-e80b-4fdb-bdb3-f70750fc292c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:34.843331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07dec9-e80b-4fdb-bdb3-f70750fc292c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-dc07dec9-e80b-4fdb-bdb3-f70750fc292c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:34.851347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:34.852253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:34.853750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 368/722] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:34.858428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5ed04-5406-4e5a-baf6-179978a8ab49 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:35.019221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:35.039933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc341cd8-2349-4b11-8b72-d42ebdea7dae None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.040372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc341cd8-2349-4b11-8b72-d42ebdea7dae None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.041136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc341cd8-2349-4b11-8b72-d42ebdea7dae None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.044346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 369/723] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:35.046273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5ed04-5406-4e5a-baf6-179978a8ab49 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:35.051815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01794022-ba1b-4b1b-8be1-b4db18604e0f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.052693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01794022-ba1b-4b1b-8be1-b4db18604e0f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.053407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01794022-ba1b-4b1b-8be1-b4db18604e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.092937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-01794022-ba1b-4b1b-8be1-b4db18604e0f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:35.094177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-01794022-ba1b-4b1b-8be1-b4db18604e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:35.103875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:35.104801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.105276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.105689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.106631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:35.107109 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:35.108667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:35.123779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 355/724] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:35.131490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:35.201908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 370/725] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:35.225662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01794022-ba1b-4b1b-8be1-b4db18604e0f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:35.300140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:35.332047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01794022-ba1b-4b1b-8be1-b4db18604e0f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:35.357852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8433d-e3e2-4eeb-92fc-564647d73e91 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:35.369871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644aff5-d3be-4f0d-9256-d9af3f60dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:35.370999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644aff5-d3be-4f0d-9256-d9af3f60dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.371487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644aff5-d3be-4f0d-9256-d9af3f60dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.372108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644aff5-d3be-4f0d-9256-d9af3f60dfa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.383430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644aff5-d3be-4f0d-9256-d9af3f60dfa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d644aff5-d3be-4f0d-9256-d9af3f60dfa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:35.392351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:35.393333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:35.394169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:35.397038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 371/726] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:35.408459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644aff5-d3be-4f0d-9256-d9af3f60dfa0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:35.439939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:35.442420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.442911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.444321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.466432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:35.467891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:35.479659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:35.502096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:35.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:35.504070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:35.504070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:35.504070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:35.504070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:35.504070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 356/727] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:35.522756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:35.549886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644aff5-d3be-4f0d-9256-d9af3f60dfa0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:35.581392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:35.586836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:35.587773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.588259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.588737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.589714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:35.590241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:35.592380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:35.602151 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 372/728] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:35.607985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:35.716074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1fabf-339d-4e2c-bc71-7a6ea7890e9b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:35.718186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468870e-5f56-47f0-9fcc-762d289e9c95 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:35.763506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:35.764674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.765184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.765779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.770402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:35.771076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.771357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.771632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.772355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:35.772663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:35.774068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:35.779533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:35.781277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:35.801191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 357/729] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:35.805088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:35.805953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:35.806829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:35.806829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:35.806829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:35.806829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:35.813807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 373/730] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:35.814439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:35.815230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:35.876842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:35.929762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:35.948401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b14d67-236b-4d15-8a62-1b579987824c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:35.988079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:35.989159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:35.989651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:35.990131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:35.991060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:35.991568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:35.993175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:36.005865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 374/731] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:36.011383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:36.031058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca98dc4-3012-4654-8665-5121178b2148 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:36.093143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a893bd3-edad-4c79-96f1-ac1597e71c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:36.094509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a893bd3-edad-4c79-96f1-ac1597e71c33 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.094939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a893bd3-edad-4c79-96f1-ac1597e71c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.095360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a893bd3-edad-4c79-96f1-ac1597e71c33 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.105755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a893bd3-edad-4c79-96f1-ac1597e71c33 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:36.107941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a893bd3-edad-4c79-96f1-ac1597e71c33 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-7a893bd3-edad-4c79-96f1-ac1597e71c33 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:36.129044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:36.130223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:36.131380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:36.134177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 358/732] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:36.135958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1b1ec-04c8-4280-9bd7-1b04ec27f466 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.136418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1b1ec-04c8-4280-9bd7-1b04ec27f466 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.136857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1b1ec-04c8-4280-9bd7-1b04ec27f466 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.142897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 359/733] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:36.145647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8526bec-a00c-480c-b32f-e7cd1bd103df None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:36.145767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1b1ec-04c8-4280-9bd7-1b04ec27f466 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:36.155078 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:36.183495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c3bad-19fa-4855-a2cc-d750b4845673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:36.184337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c3bad-19fa-4855-a2cc-d750b4845673 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.184626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c3bad-19fa-4855-a2cc-d750b4845673 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.184906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c3bad-19fa-4855-a2cc-d750b4845673 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.193063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c3bad-19fa-4855-a2cc-d750b4845673 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:36.194547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c3bad-19fa-4855-a2cc-d750b4845673 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2f7c3bad-19fa-4855-a2cc-d750b4845673 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:36.209627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:36.210888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:36.212065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:36.215280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 375/734] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:17:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:36.217130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da46b2b-bae8-455e-ad4d-33593786fd1f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.217421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da46b2b-bae8-455e-ad4d-33593786fd1f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.217701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da46b2b-bae8-455e-ad4d-33593786fd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.234777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4da46b2b-bae8-455e-ad4d-33593786fd1f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:36.235951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4da46b2b-bae8-455e-ad4d-33593786fd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:36.271994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1b1ec-04c8-4280-9bd7-1b04ec27f466 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:36.288433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:36.302295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 376/735] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:36.303592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449464c-3ac0-4ab6-8b31-a76c457ded2e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.303905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449464c-3ac0-4ab6-8b31-a76c457ded2e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.304189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449464c-3ac0-4ab6-8b31-a76c457ded2e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.305300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fc021-d089-4371-87f0-b298a3feb860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:36.305721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 377/736] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:36.306317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc021-d089-4371-87f0-b298a3feb860 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.306419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449464c-3ac0-4ab6-8b31-a76c457ded2e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:36.306908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc021-d089-4371-87f0-b298a3feb860 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.307190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc021-d089-4371-87f0-b298a3feb860 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.308022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fc021-d089-4371-87f0-b298a3feb860 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:36.308333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fc021-d089-4371-87f0-b298a3feb860 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:36.310116 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fc021-d089-4371-87f0-b298a3feb860 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:36.331865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 360/737] 10.222.0.41 () {56 vars in 1135 bytes} [Sat May 28 03:17:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:36.333179 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109845b0-25a7-4324-bf41-7186d5da9052 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.333477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109845b0-25a7-4324-bf41-7186d5da9052 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.333761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109845b0-25a7-4324-bf41-7186d5da9052 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.337236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 361/738] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:36.338591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee1a88-112f-42eb-8cad-5aa2c79d8f7b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.338937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee1a88-112f-42eb-8cad-5aa2c79d8f7b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.339542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee1a88-112f-42eb-8cad-5aa2c79d8f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.356435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-45ee1a88-112f-42eb-8cad-5aa2c79d8f7b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:36.357666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-45ee1a88-112f-42eb-8cad-5aa2c79d8f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:36.404733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:36.411630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449464c-3ac0-4ab6-8b31-a76c457ded2e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:36.426660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 362/739] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:36.427564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee1a88-112f-42eb-8cad-5aa2c79d8f7b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:36.451675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b1726-54be-4245-981a-a165772af443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:36.452341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b1726-54be-4245-981a-a165772af443 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.452620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b1726-54be-4245-981a-a165772af443 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.452895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b1726-54be-4245-981a-a165772af443 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.453819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b1726-54be-4245-981a-a165772af443 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:36.454126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b1726-54be-4245-981a-a165772af443 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:36.455551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b1726-54be-4245-981a-a165772af443 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:36.472568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 378/740] 10.222.0.41 () {56 vars in 1149 bytes} [Sat May 28 03:17:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:36.473375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b1726-54be-4245-981a-a165772af443 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:36.525793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee1a88-112f-42eb-8cad-5aa2c79d8f7b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:36.532797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:36.557654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64461c-ffa0-4743-9e62-933c691364c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:36.558433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64461c-ffa0-4743-9e62-933c691364c9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.558719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64461c-ffa0-4743-9e62-933c691364c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.558992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64461c-ffa0-4743-9e62-933c691364c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.569145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64461c-ffa0-4743-9e62-933c691364c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:36.571522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b1726-54be-4245-981a-a165772af443 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3c64461c-ffa0-4743-9e62-933c691364c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:36.577322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:36.578283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:36.579121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:36.583512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 363/741] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:36.584872 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215c69-3c02-4a86-8931-ff4cc933ba83 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.585688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215c69-3c02-4a86-8931-ff4cc933ba83 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.586332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe215c69-3c02-4a86-8931-ff4cc933ba83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.603037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-fe215c69-3c02-4a86-8931-ff4cc933ba83 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:36.604200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fe215c69-3c02-4a86-8931-ff4cc933ba83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:36.605301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b00f2-7952-469e-9d5d-cd8d13896a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:36.605949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b00f2-7952-469e-9d5d-cd8d13896a2d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.606261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b00f2-7952-469e-9d5d-cd8d13896a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.606544 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b00f2-7952-469e-9d5d-cd8d13896a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.630255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:36.670685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 364/742] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 03:17:36.671270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe215c69-3c02-4a86-8931-ff4cc933ba83 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:36.676677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b00f2-7952-469e-9d5d-cd8d13896a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:36.678169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b00f2-7952-469e-9d5d-cd8d13896a2d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:36.777119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:36.777844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 379/743] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:36.778390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b00f2-7952-469e-9d5d-cd8d13896a2d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:36.789938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe215c69-3c02-4a86-8931-ff4cc933ba83 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:36.824115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf5d48-974d-4d7c-9e96-e7eb5cdcd0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:36.824771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf5d48-974d-4d7c-9e96-e7eb5cdcd0da None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.825054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf5d48-974d-4d7c-9e96-e7eb5cdcd0da None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.825361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf5d48-974d-4d7c-9e96-e7eb5cdcd0da None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.826071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf5d48-974d-4d7c-9e96-e7eb5cdcd0da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:36.827434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf5d48-974d-4d7c-9e96-e7eb5cdcd0da None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:36.832183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:36.879345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b00f2-7952-469e-9d5d-cd8d13896a2d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:36.912469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63d58f-4fd5-4047-8a4e-a6d34b788125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:36.913142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63d58f-4fd5-4047-8a4e-a6d34b788125 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:36.913428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63d58f-4fd5-4047-8a4e-a6d34b788125 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:36.913695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63d58f-4fd5-4047-8a4e-a6d34b788125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:36.923969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 365/744] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:17:36 2022] POST /identity/v3/services => generated 240 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:36.923969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf5d48-974d-4d7c-9e96-e7eb5cdcd0da None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:36.997704 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63d58f-4fd5-4047-8a4e-a6d34b788125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:36.999110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63d58f-4fd5-4047-8a4e-a6d34b788125 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.058326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf5d48-974d-4d7c-9e96-e7eb5cdcd0da None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.067743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:37.100329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.101406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.101873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.102746 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.103692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:37.104004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.105422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.115285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 366/745] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:37.119077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.119216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 380/746] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 03:17:37.119216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63d58f-4fd5-4047-8a4e-a6d34b788125 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.133429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:37.226474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63d58f-4fd5-4047-8a4e-a6d34b788125 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.227304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf6776-6e48-40bc-9c22-f0bd89f026e0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.264070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.264859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.265159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.265447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.269982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e0ea-2973-4b70-a7ed-0ff9038f1cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.270812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e0ea-2973-4b70-a7ed-0ff9038f1cbf None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.271263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e0ea-2973-4b70-a7ed-0ff9038f1cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.271684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef80e0ea-2973-4b70-a7ed-0ff9038f1cbf None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.272920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80e0ea-2973-4b70-a7ed-0ff9038f1cbf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.275146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef80e0ea-2973-4b70-a7ed-0ff9038f1cbf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.275529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.276951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:37.295896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:37.297017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:37.298045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:37.298045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:37.298045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:37.298045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:37.298045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 381/747] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:37.298314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.307244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:37.361411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 367/748] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:17:37 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:37.362476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e0ea-2973-4b70-a7ed-0ff9038f1cbf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.419493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe79b07-fd05-494c-b04d-9a5dc0b7a6f1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.452788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:37.488434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef80e0ea-2973-4b70-a7ed-0ff9038f1cbf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.489553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.491739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.492274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.492781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.508659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.510712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.527488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.528837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.529138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.529427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:37.537821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:37.538736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:37.539615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:37.539615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:37.539615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:37.539615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:37.542659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.542906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 382/749] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:37.544544 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.544741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:37.564573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:37.565577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:37.567043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:37.567043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:37.567043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:37.567043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:37.567043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 368/750] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:37.567043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.613968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:37.664943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d217b-3ebf-4c31-bbf0-da567a9e4816 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.693968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea579d-97a8-42b5-aaef-e51edcf6279b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.704808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.705507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.706107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.706390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.707400 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:37.707709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.709633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.718019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:37.725585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 383/751] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:37.728283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.732856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.733068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.733385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.733664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.734455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:37.734765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.736215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.751781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 369/752] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:37.756271 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.851573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba0fda-809c-4db3-addf-994843691b07 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.863137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c872ba-2ff6-4426-a193-a736987c5823 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:37.867080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:37.888438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.893623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.893623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.893623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.893623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:37.893623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.893623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.900198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:37.901004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:37.909781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:37.910197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 384/753] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:37.910415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:37.911097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.919318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:37.920782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:37.936206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:37.937139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 370/754] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:37.938425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:37.991279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:38.033436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fac72-a0fb-48b8-85dc-c6f6f2580bd1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:38.064757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb342130-3f3e-4ea9-8097-3c5162992f42 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:38.072573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:38.074046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:38.074488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:38.074997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:38.085376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:38.088179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:38.103092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:38.103977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:38.104331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:38.104665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:38.109724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:38.110805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:38.111935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:38.111935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:38.111935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:38.111935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:38.111935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:38.111935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:38.111935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:38.111935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:38.114459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:38.114625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 385/755] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:38.115552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:38.115885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:38.136095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:38.137437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:38.138920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:38.142353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 371/756] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:17:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:38.143199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:38.161746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:38.230904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2008c0-14f3-4f4e-94a5-621378d1de3e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:38.268824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:38.269503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:38.269789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:38.270059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:38.270828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:38.271133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:38.272609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:38.277168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82aede-30c5-4e1e-ab0e-7a9733b1b46b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:38.278953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:38.290111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 386/757] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:38.290111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:38.319577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:38.320722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:38.321795 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:38.322071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:38.322842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:38.323149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:38.324585 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:38.341581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 372/758] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:38.342426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:38.420299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:38.429881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75444f2f-c96c-4a93-b0ae-8b83ac1af901 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:38.449864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8d15b-33bc-4cd4-87ac-371acf93b034 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:38.477553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:38.478827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:38.479610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:38.480515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:38.482351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:38.482880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:38.484512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:38.485401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:38.486048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:38.486328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:38.486601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:38.487480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f2f14380f60f4fa18ed6a9579cfb071b) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:38.487791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:38.489423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:38.501134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 373/759] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:38 2022] GET /identity/v3/role_assignments?user.id=f2f14380f60f4fa18ed6a9579cfb071b&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=bb0756f8af694633b9743a958eb46e7a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:38.501134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:38.526164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 387/760] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:17:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:38.528225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:38.561198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:38.698154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7c35b-f8a2-4fdd-bee4-24b26e82cdcb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:38.742939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d02c1c-ba73-4092-bd56-c498789b91b7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:38.750775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:38.751923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:38.752214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:38.752494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:38.753598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=e619c4b1fa304a8a99de2f60d1b6cc89) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:38.753731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:38.755116 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:38.766662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 374/761] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:38 2022] GET /identity/v3/role_assignments?user.id=e619c4b1fa304a8a99de2f60d1b6cc89&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:38.790859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:38.791706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:38.791996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:38.792279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:38.810065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:38.811628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:38.832485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:38.833494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:38.834397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:38.834397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:38.834397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:38.834397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:38.834397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 388/762] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:38.842313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:38.978383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:38.997995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff9450-2d89-40e2-94f4-5fc0dda7d1ce None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:39.048754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450496d-420f-41dc-845c-7dc275e331e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:39.049453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450496d-420f-41dc-845c-7dc275e331e2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.049737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450496d-420f-41dc-845c-7dc275e331e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.050014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450496d-420f-41dc-845c-7dc275e331e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:39.050735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450496d-420f-41dc-845c-7dc275e331e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:39.051044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450496d-420f-41dc-845c-7dc275e331e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:39.052480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450496d-420f-41dc-845c-7dc275e331e2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:39.078388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 375/763] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:39.083468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:39.103952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:39.358472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f240-c69f-4d8d-afbe-e3585f2537b3 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.358650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f240-c69f-4d8d-afbe-e3585f2537b3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.358932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1f240-c69f-4d8d-afbe-e3585f2537b3 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:39.361945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 376/764] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:39.365755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9517-c45d-4a6a-a631-4082667be0c7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.366018 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9517-c45d-4a6a-a631-4082667be0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.366297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9517-c45d-4a6a-a631-4082667be0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:39.385253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d04a0-3094-4bd3-96cc-63f7b0936d06 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:39.403727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1cbc9517-c45d-4a6a-a631-4082667be0c7 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:39.404911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1cbc9517-c45d-4a6a-a631-4082667be0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:39.426567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:39.427708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.428208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.428684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:39.439691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:39.441591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:39.467774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:39.468696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:39.469630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:39.469630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:39.469630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:39.469630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:39.469630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:39.469630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:39.469630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:39.469630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:39.474617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 389/765] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:39.482543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:39.518042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 377/766] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:17:39.562561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc9517-c45d-4a6a-a631-4082667be0c7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:39.586939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:39.632778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a781176-c553-47b2-9a2f-0ab4c0caffeb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:39.694279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:39.695907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.696595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.698162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:39.699866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:39.701026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:39.702876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:39.723943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 390/767] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:39.726703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:39.758994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:39.811258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc9517-c45d-4a6a-a631-4082667be0c7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:39.840336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a777f4d-7856-44b5-9793-6e933d1ccf2a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:39.866641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:39.879405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef1e1c-d665-4039-814a-46b221de0de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:39.880588 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef1e1c-d665-4039-814a-46b221de0de8 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.880865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef1e1c-d665-4039-814a-46b221de0de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.881538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef1e1c-d665-4039-814a-46b221de0de8 None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f7ebbd8eb36c4919aedd3498bd8bc483/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:39.884007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef1e1c-d665-4039-814a-46b221de0de8 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=f7ebbd8eb36c4919aedd3498bd8bc483, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:39.885357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc406b-8380-481d-905b-8f2b2e6a5c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:39.885501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef1e1c-d665-4039-814a-46b221de0de8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:39.886154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc406b-8380-481d-905b-8f2b2e6a5c37 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.886434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc406b-8380-481d-905b-8f2b2e6a5c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.886705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc406b-8380-481d-905b-8f2b2e6a5c37 None admin] PATH_INFO: `/v3/services/network` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:39.887890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc406b-8380-481d-905b-8f2b2e6a5c37 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:39.889271 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc406b-8380-481d-905b-8f2b2e6a5c37 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-39dc406b-8380-481d-905b-8f2b2e6a5c37 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:39.901434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:39.908615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:39.909474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 378/768] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:39 2022] GET /identity/v3/services/network => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:39.910274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd3946-1aa1-495b-bf88-591b3bfb05c7 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.910274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd3946-1aa1-495b-bf88-591b3bfb05c7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.910274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd3946-1aa1-495b-bf88-591b3bfb05c7 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:39.910274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 379/769] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:39.910274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd3946-1aa1-495b-bf88-591b3bfb05c7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:39.965321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 391/770] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:17:39 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f7ebbd8eb36c4919aedd3498bd8bc483/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:17:39.967411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b48f6-652e-4e8d-9c2a-e3f3abfa51c7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:39.967701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b48f6-652e-4e8d-9c2a-e3f3abfa51c7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:39.967981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b48f6-652e-4e8d-9c2a-e3f3abfa51c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:40.026196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-503b48f6-652e-4e8d-9c2a-e3f3abfa51c7 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:40.027610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-503b48f6-652e-4e8d-9c2a-e3f3abfa51c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:40.044984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd3946-1aa1-495b-bf88-591b3bfb05c7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:40.089403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:40.091074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:40.091748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:40.092281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:40.093282 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] RBAC: Adding query filter params (name=network) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:40.093804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:40.095533 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:40.102547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 380/771] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:17:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:40.107867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:40.132394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:40.207393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 392/772] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:40.230789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b48f6-652e-4e8d-9c2a-e3f3abfa51c7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:40.268495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992af47-6d19-4759-9ed0-3c124bea8d11 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:40.304535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:40.305237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:40.305560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:40.305862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:40.306545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] RBAC: Adding query filter params (type=network) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:40.306851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:40.308333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:40.317240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 381/773] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:17:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:40.319634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:40.341021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b48f6-652e-4e8d-9c2a-e3f3abfa51c7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:40.355965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:40.380936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:40.381956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:40.382392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:40.382777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:40.383533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:40.385243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:40.396211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:40.397632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:40.398831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:40.399852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:40.401219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 393/774] 10.222.0.41 () {56 vars in 1129 bytes} [Sat May 28 03:17:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:40.407743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:40.433479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b9572-c3f7-4b04-b789-9ed4b61f633f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:40.483658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:40.483658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:40.483658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:40.483658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:40.484686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] RBAC: Adding query filter params (interface=public, service_id=50cf0e938c844e6b9cae90b5ed45f24b, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:40.485217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:40.488477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:40.507358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 382/775] 10.222.0.41 () {56 vars in 1251 bytes} [Sat May 28 03:17:40 2022] GET /identity/v3/endpoints?service_id=50cf0e938c844e6b9cae90b5ed45f24b&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:40.512834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:40.613427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc58ef-7170-4fa2-87d0-ee3f9371fcd8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:40.648325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:40.649347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:40.649475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:40.649774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:40.650553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:40.650932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:40.652361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:40.653731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1cf2-633e-48cb-9784-4e5e77f489cf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:40.658051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 394/776] 10.222.0.41 () {56 vars in 1124 bytes} [Sat May 28 03:17:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:40.663463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:40.701867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bd58a-b180-4d0a-9a5f-217ffd290448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:40.702974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd58a-b180-4d0a-9a5f-217ffd290448 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:40.703511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd58a-b180-4d0a-9a5f-217ffd290448 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:40.703997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6bd58a-b180-4d0a-9a5f-217ffd290448 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:40.704924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bd58a-b180-4d0a-9a5f-217ffd290448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:40.706792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6bd58a-b180-4d0a-9a5f-217ffd290448 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:40.713354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:40.722764 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 383/777] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:17:40 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:40.723671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:40.786136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f0a68-7639-4685-a19e-c5150fdc6b8a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:40.823720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:40.824450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:40.824737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:40.825582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:40.826309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:40.826617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:40.828013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:40.835294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 395/778] 10.222.0.41 () {56 vars in 1124 bytes} [Sat May 28 03:17:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:40.841838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bd58a-b180-4d0a-9a5f-217ffd290448 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:40.994193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6bd58a-b180-4d0a-9a5f-217ffd290448 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:41.031959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c7ebf-4bf8-42e7-a6ea-f4568e27d7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:41.032962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c7ebf-4bf8-42e7-a6ea-f4568e27d7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.033461 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c7ebf-4bf8-42e7-a6ea-f4568e27d7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.033931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c7ebf-4bf8-42e7-a6ea-f4568e27d7a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.034940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c7ebf-4bf8-42e7-a6ea-f4568e27d7a3 None admin] RBAC: Adding query filter params (interface=public, service_id=4bdd0f93d55f4d819e0d3e4b8a2beb26, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:41.035462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c7ebf-4bf8-42e7-a6ea-f4568e27d7a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:41.037097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c7ebf-4bf8-42e7-a6ea-f4568e27d7a3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:41.043572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 384/779] 10.222.0.41 () {56 vars in 1251 bytes} [Sat May 28 03:17:40 2022] GET /identity/v3/endpoints?service_id=4bdd0f93d55f4d819e0d3e4b8a2beb26&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:41.048771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:41.186191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade54310-93da-47fa-87c9-2223583d8b0d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:41.193268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:41.222538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2255ff-037a-4fe9-8977-19361cd440fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:41.223211 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2255ff-037a-4fe9-8977-19361cd440fc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.223499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2255ff-037a-4fe9-8977-19361cd440fc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.223775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2255ff-037a-4fe9-8977-19361cd440fc None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.224462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2255ff-037a-4fe9-8977-19361cd440fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:41.225865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2255ff-037a-4fe9-8977-19361cd440fc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:41.236433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 396/780] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:17:41 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:41.252884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c88599-9044-49a8-aa03-103577737ed2 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.253223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c88599-9044-49a8-aa03-103577737ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.253499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c88599-9044-49a8-aa03-103577737ed2 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.255219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 385/781] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:41.261258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca37ce-2aec-45a7-b67f-05c35bf988a0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.261595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca37ce-2aec-45a7-b67f-05c35bf988a0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.261838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca37ce-2aec-45a7-b67f-05c35bf988a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.298428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f9ca37ce-2aec-45a7-b67f-05c35bf988a0 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:41.299574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f9ca37ce-2aec-45a7-b67f-05c35bf988a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:41.388799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 397/782] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:41.411856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c88599-9044-49a8-aa03-103577737ed2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:41.474987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3cafd-a2b2-4f35-991b-75ef79071f0d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.475296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3cafd-a2b2-4f35-991b-75ef79071f0d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.475549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3cafd-a2b2-4f35-991b-75ef79071f0d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.481857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 398/783] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:41.483315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fa5d1-14ae-46c5-8cec-ab74a0ce90a1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.483603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fa5d1-14ae-46c5-8cec-ab74a0ce90a1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.483876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fa5d1-14ae-46c5-8cec-ab74a0ce90a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.493540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:41.515412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-601fa5d1-14ae-46c5-8cec-ab74a0ce90a1 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:41.516973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-601fa5d1-14ae-46c5-8cec-ab74a0ce90a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:41.550551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c88599-9044-49a8-aa03-103577737ed2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:41.594324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c31d4f-88b1-4c6a-88a9-2a6d42a72912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:41.595560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31d4f-88b1-4c6a-88a9-2a6d42a72912 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.597765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31d4f-88b1-4c6a-88a9-2a6d42a72912 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.598543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31d4f-88b1-4c6a-88a9-2a6d42a72912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.609851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 399/784] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:41.618604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c31d4f-88b1-4c6a-88a9-2a6d42a72912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-71c31d4f-88b1-4c6a-88a9-2a6d42a72912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:41.627442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:41.628336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:41.629204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:41.631944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 386/785] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:41.639195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fa5d1-14ae-46c5-8cec-ab74a0ce90a1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:41.640535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c31d4f-88b1-4c6a-88a9-2a6d42a72912 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:41.669875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:41.812847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fa5d1-14ae-46c5-8cec-ab74a0ce90a1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:41.826688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c31d4f-88b1-4c6a-88a9-2a6d42a72912 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:41.869477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:41.880143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:41.888934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.888934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.888934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.888934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:41.888934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:41.888934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:41.890894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d816e2f-225c-46d5-8f27-a523f1893bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:41.891693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d816e2f-225c-46d5-8f27-a523f1893bcf None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:41.891961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d816e2f-225c-46d5-8f27-a523f1893bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:41.892230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d816e2f-225c-46d5-8f27-a523f1893bcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:41.904885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 400/786] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:41.909817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:41.915773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d816e2f-225c-46d5-8f27-a523f1893bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0d816e2f-225c-46d5-8f27-a523f1893bcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:41.932852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:41.933882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:41.935323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:41.937529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 387/787] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:41.943505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d816e2f-225c-46d5-8f27-a523f1893bcf None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:42.064472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:42.075402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd69808-80f6-46f3-9c10-20ef7f149f1a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:42.091624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d816e2f-225c-46d5-8f27-a523f1893bcf None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:42.128324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:42.129134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:42.129426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:42.129699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:42.148716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:42.150185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:42.156035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:42.156709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:42.156995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:42.157295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:42.158033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:42.158338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:42.159705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:42.179049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 388/788] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:42.183285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:42.184220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:42.185091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:42.185091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:42.185091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:42.185091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:42.185786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:42.190127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 401/789] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:42.190275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:42.264509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:42.371344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e51b4-4c00-447d-8c57-d7b19c146467 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:42.376661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa76fa-e3eb-4a37-acda-cbd16c540193 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:42.413845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:42.414502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:42.414785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:42.415060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:42.416045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:42.416350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:42.417897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:42.431425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 402/790] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:42.436926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:42.442212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:42.443894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:42.444687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:42.444965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:42.445267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:42.462626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:42.464132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:42.481935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:42.483035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:42.484128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:42.484128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:42.484128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:42.484128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:42.484128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 389/791] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:42.489542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:42.612328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c7ad2-7b64-452b-a1a5-d6ebd0380f79 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:42.662772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:42.667842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f70bbf-02aa-4f9c-a43e-6516e68f86ce None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:42.678112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:42.678913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:42.679193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:42.679467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:42.693029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:42.694543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:42.722458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:42.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:42.724475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 403/792] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:42.730155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:42.751355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:42.752035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:42.752314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:42.752582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:42.753333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:42.753647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:42.755043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:42.779711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 390/793] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:42.785720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:42.853499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:42.932942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ab4c-4a8c-41d3-9efe-783a1e8d9d5f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:42.940679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ae78-3d45-496f-97cd-63f5b156c198 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:42.977778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b5986-6d38-4faf-804c-29d0c7a4900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:42.978802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b5986-6d38-4faf-804c-29d0c7a4900e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:42.979261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b5986-6d38-4faf-804c-29d0c7a4900e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:42.979680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b5986-6d38-4faf-804c-29d0c7a4900e None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:42.980712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b5986-6d38-4faf-804c-29d0c7a4900e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:42.981233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b5986-6d38-4faf-804c-29d0c7a4900e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:42.982353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48455ce2-cadd-4726-8e44-53d9b36839e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:42.983078 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b5986-6d38-4faf-804c-29d0c7a4900e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:42.983342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48455ce2-cadd-4726-8e44-53d9b36839e0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:42.983717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48455ce2-cadd-4726-8e44-53d9b36839e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:42.984150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48455ce2-cadd-4726-8e44-53d9b36839e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:42.997912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48455ce2-cadd-4726-8e44-53d9b36839e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:42.999562 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48455ce2-cadd-4726-8e44-53d9b36839e0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:43.002513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 404/794] 10.222.0.41 () {56 vars in 1135 bytes} [Sat May 28 03:17:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:43.004032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2f7c-120c-473b-b41a-feebdf8f9510 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.004471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2f7c-120c-473b-b41a-feebdf8f9510 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.004896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c2f7c-120c-473b-b41a-feebdf8f9510 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.006534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 405/795] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:43.007431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2f7c-120c-473b-b41a-feebdf8f9510 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-48455ce2-cadd-4726-8e44-53d9b36839e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:43.020306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:43.021323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:43.022243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 391/796] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:17:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:43.023321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b5e6c-d849-45f5-91f3-17f05b9b14ac None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.023427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b5e6c-d849-45f5-91f3-17f05b9b14ac None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.023694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b5e6c-d849-45f5-91f3-17f05b9b14ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.040855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b45b5e6c-d849-45f5-91f3-17f05b9b14ac None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:43.042073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b45b5e6c-d849-45f5-91f3-17f05b9b14ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:43.055666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:43.112779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 392/797] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:43.114037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b5e6c-d849-45f5-91f3-17f05b9b14ac None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.158066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c2f7c-120c-473b-b41a-feebdf8f9510 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.189516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:43.204274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:43.205795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.206186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.206608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.220599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:43.222566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:43.234945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b5e6c-d849-45f5-91f3-17f05b9b14ac None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:43.242180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:43.243264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:43.244163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:43.244163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:43.244163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:43.244163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:43.246253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 406/798] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:43.247314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.274626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:43.275649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.276167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.276623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.277672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:43.278659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:43.280241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:43.300169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 393/799] 10.222.0.41 () {56 vars in 1149 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:43.301287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.310095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:43.352733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601357f-0f9f-4157-b9b8-749c3f1963d8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.386901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9befab-6052-4d12-9a24-ca48756d647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:43.387723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9befab-6052-4d12-9a24-ca48756d647b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.388004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9befab-6052-4d12-9a24-ca48756d647b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.388308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9befab-6052-4d12-9a24-ca48756d647b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.400236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9befab-6052-4d12-9a24-ca48756d647b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:43.405160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8af5ac-d48a-4a3d-9c5c-bb2021987002 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1b9befab-6052-4d12-9a24-ca48756d647b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:43.408431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:43.409416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:43.410248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 407/800] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:43.411400 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d70479-76ee-410e-a995-12d4f587050b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.411497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d70479-76ee-410e-a995-12d4f587050b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.411764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d70479-76ee-410e-a995-12d4f587050b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.412996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 408/801] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:43.413765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d70479-76ee-410e-a995-12d4f587050b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.454775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:43.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:43.455665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.456712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.456712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.457657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:43.458140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:43.460327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:43.482648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 394/802] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:43.482648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.548691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d70479-76ee-410e-a995-12d4f587050b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.567828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:43.585219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5e9eb-a5c4-477f-ae02-7258afa73586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:43.586170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5e9eb-a5c4-477f-ae02-7258afa73586 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.586518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5e9eb-a5c4-477f-ae02-7258afa73586 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.586857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5e9eb-a5c4-477f-ae02-7258afa73586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.599337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5e9eb-a5c4-477f-ae02-7258afa73586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:43.600805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5e9eb-a5c4-477f-ae02-7258afa73586 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:43.604616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dab057-3f5f-41bc-9579-a642be4ac462 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-60d5e9eb-a5c4-477f-ae02-7258afa73586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:43.618284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:43.619168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:43.620765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:43.620765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:43.620765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:43.620765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:43.620765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 409/803] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:43.621037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603c471-09e3-4748-ad06-63755783ec4c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.621644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603c471-09e3-4748-ad06-63755783ec4c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.621644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603c471-09e3-4748-ad06-63755783ec4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.639074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4603c471-09e3-4748-ad06-63755783ec4c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:43.640252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4603c471-09e3-4748-ad06-63755783ec4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:43.644804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:43.645665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.646115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.646534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.647412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:43.647870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:43.649261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:43.658433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 395/804] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:43.659251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.714797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 410/805] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:43.716189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603c471-09e3-4748-ad06-63755783ec4c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.742400 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:43.782727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cace8e-21dc-43bd-a016-0358c0593aa8 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.814899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:43.823258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:43.824189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.824431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.825298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.831987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:43.832110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603c471-09e3-4748-ad06-63755783ec4c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.833474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:43.849155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:43.850201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:43.851106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:43.851106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:43.851106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:43.851106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:43.851106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:43.851106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:43.851106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:43.851106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:43.859142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 396/806] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:43.859947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.872474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:43.873161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:43.873427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:43.873701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:43.874420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:43.874725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:43.876084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:43.888326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 411/807] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:43.889450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:43.967376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:43.971640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5dee48-5b5a-4f66-a1c2-7f4811f39048 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:43.988995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f33918-0a16-4584-85b9-d29dba1064c9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.005189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.005979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.006266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.006546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.016962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.018446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.023318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.024098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.024382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.024654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] PATH_INFO: `/v3/services/network` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.025333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.026679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:44.035452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:44.036522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:44.037779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:44.037779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:44.037779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:44.037779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:44.038155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:44.039301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:44.040801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 412/808] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/services/network => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:44.042003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.042087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 397/809] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:44.042170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.083152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:44.139471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dffeea-d304-4b72-8c73-ef622cf87a39 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.140749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08915-5515-4b74-b99f-b2015617a403 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.172126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.172707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e7ac7-23c7-4d57-b766-6e7e25a2d952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.173218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.173218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.173387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e7ac7-23c7-4d57-b766-6e7e25a2d952 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.173470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.173655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e7ac7-23c7-4d57-b766-6e7e25a2d952 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.173942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e7ac7-23c7-4d57-b766-6e7e25a2d952 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.174719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e7ac7-23c7-4d57-b766-6e7e25a2d952 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:44.175032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e7ac7-23c7-4d57-b766-6e7e25a2d952 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.176458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e7ac7-23c7-4d57-b766-6e7e25a2d952 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.179962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.181414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.186031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:44.189320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 413/810] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:44.190526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b45b82-ab5a-4d3d-adec-dde90371f63e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.190817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b45b82-ab5a-4d3d-adec-dde90371f63e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.191100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b45b82-ab5a-4d3d-adec-dde90371f63e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.192483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 414/811] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:44.194045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b45b82-ab5a-4d3d-adec-dde90371f63e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:44.195293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:44.196994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 398/812] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:44.197906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.294507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:44.297291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b24b4-7de2-452f-bceb-9a878e5c645e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.297725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b45b82-ab5a-4d3d-adec-dde90371f63e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.330430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.331096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.331441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde87e6b-d263-4aa2-afd9-52a4604557ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.331581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.331840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.332093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde87e6b-d263-4aa2-afd9-52a4604557ec None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.332379 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde87e6b-d263-4aa2-afd9-52a4604557ec None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.332660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde87e6b-d263-4aa2-afd9-52a4604557ec None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.332826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] RBAC: Adding query filter params (name=network) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:44.333125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.333412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde87e6b-d263-4aa2-afd9-52a4604557ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:44.333725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde87e6b-d263-4aa2-afd9-52a4604557ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.334512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.335066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde87e6b-d263-4aa2-afd9-52a4604557ec None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.338942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 399/813] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:17:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:44.339689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.346792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 415/814] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:44.348311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7755-69a3-43cb-a60f-b41f540ffba4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.348569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7755-69a3-43cb-a60f-b41f540ffba4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.348844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7755-69a3-43cb-a60f-b41f540ffba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.365063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-df0f7755-69a3-43cb-a60f-b41f540ffba4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:44.366280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-df0f7755-69a3-43cb-a60f-b41f540ffba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:44.420550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:44.431221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 416/815] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:44.438732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f7755-69a3-43cb-a60f-b41f540ffba4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.439374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de131a-d0d8-49b3-ba2e-526edccd0926 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.473263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaea6b-dc84-450a-a1f1-1408c4f4d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.474138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaea6b-dc84-450a-a1f1-1408c4f4d192 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.474416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaea6b-dc84-450a-a1f1-1408c4f4d192 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.474688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaea6b-dc84-450a-a1f1-1408c4f4d192 None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f2f14380f60f4fa18ed6a9579cfb071b/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.476603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdaea6b-dc84-450a-a1f1-1408c4f4d192 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=f2f14380f60f4fa18ed6a9579cfb071b, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.478058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdaea6b-dc84-450a-a1f1-1408c4f4d192 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.542401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f7755-69a3-43cb-a60f-b41f540ffba4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.553612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 400/816] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:17:44 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f2f14380f60f4fa18ed6a9579cfb071b/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:17:44.554347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaea6b-dc84-450a-a1f1-1408c4f4d192 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.582844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:44.587792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.588476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.588757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.589024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.589869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:44.590162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.592357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.606737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 417/817] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:44.606737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.672510 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:44.674875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaea6b-dc84-450a-a1f1-1408c4f4d192 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.715077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.716164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.716658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.717645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.718848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] RBAC: Adding query filter params (type=network) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:44.719176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.721122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.727454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 401/818] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:17:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:44.728383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.738738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ade015-e93a-458f-832d-b89e15a116e7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.777308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.778219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.778501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.778775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.786741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.788177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:44.802710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:44.803682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 418/819] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:17:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:44.804880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.835607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:44.848745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222196-8c12-43be-8698-f18fcdb69194 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.884407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.885314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.885701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.885980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.886724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.888744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:44.901748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:44.907811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:44.908676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 402/820] 10.222.0.41 () {56 vars in 1129 bytes} [Sat May 28 03:17:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:44.910178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:44.919487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8bcd2-922b-4b11-9371-ff7c4845bf04 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:44.930847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:44.954930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47479d95-b55e-49fc-aeef-08564ae1e246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:44.955815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47479d95-b55e-49fc-aeef-08564ae1e246 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:44.956094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47479d95-b55e-49fc-aeef-08564ae1e246 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:44.956367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47479d95-b55e-49fc-aeef-08564ae1e246 None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/e619c4b1fa304a8a99de2f60d1b6cc89/roles/e648d64a4d3d440ba9feaa50a398c2e6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:44.973019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47479d95-b55e-49fc-aeef-08564ae1e246 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=e619c4b1fa304a8a99de2f60d1b6cc89, role_id=e648d64a4d3d440ba9feaa50a398c2e6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:44.974694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47479d95-b55e-49fc-aeef-08564ae1e246 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:45.017167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed162263-0cbb-4de4-87d6-7b4f72cc66de None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:45.038882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 419/821] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:17:44 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/e619c4b1fa304a8a99de2f60d1b6cc89/roles/e648d64a4d3d440ba9feaa50a398c2e6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:17:45.040809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47479d95-b55e-49fc-aeef-08564ae1e246 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:45.052571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d7bb8-d3c9-490a-a4b0-7cd2941fc6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:45.053293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d7bb8-d3c9-490a-a4b0-7cd2941fc6db None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:45.053582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d7bb8-d3c9-490a-a4b0-7cd2941fc6db None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:45.053943 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d7bb8-d3c9-490a-a4b0-7cd2941fc6db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:45.054608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d7bb8-d3c9-490a-a4b0-7cd2941fc6db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:45.056022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d7bb8-d3c9-490a-a4b0-7cd2941fc6db None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:45.141595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 403/822] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:17:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:45.142717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d7bb8-d3c9-490a-a4b0-7cd2941fc6db None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:45.168976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:45.177205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47479d95-b55e-49fc-aeef-08564ae1e246 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:45.234570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:45.235745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:45.236278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:45.236811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:45.238204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:45.238753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:45.240672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:45.270909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 420/823] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:17:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:45.279176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:45.302231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:45.336441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d7bb8-d3c9-490a-a4b0-7cd2941fc6db None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:45.381793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:45.383940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:45.384441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:45.384915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:45.385868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:45.386377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:45.388847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:45.396681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 404/824] 10.222.0.41 () {56 vars in 1124 bytes} [Sat May 28 03:17:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:45.402680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:45.423231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4550e0aa-c848-4d41-9959-8d45fd35d16c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:45.461715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2898e-058b-43b3-934e-3920e9f03068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:45.462737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2898e-058b-43b3-934e-3920e9f03068 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:45.463234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2898e-058b-43b3-934e-3920e9f03068 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:45.463729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2898e-058b-43b3-934e-3920e9f03068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:45.493648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:45.532491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7bb38-7b84-46f6-9dfa-2ba3e31abfdc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:45.540379 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2898e-058b-43b3-934e-3920e9f03068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:45.542184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2898e-058b-43b3-934e-3920e9f03068 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:45.559263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:45.575159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:45.576119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:45.576420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:45.576696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:45.577693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:45.578008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:45.579458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:45.585469 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 405/825] 10.222.0.41 () {56 vars in 1124 bytes} [Sat May 28 03:17:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:45.592988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:45.665721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 421/826] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:17:45.678296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2898e-058b-43b3-934e-3920e9f03068 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:45.718136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769eea5-c55c-4b37-90ed-1f46381d5ed4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:45.762181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3233f4-8815-4616-a8ef-ef0b995704ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:45.762856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3233f4-8815-4616-a8ef-ef0b995704ac None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:45.763143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3233f4-8815-4616-a8ef-ef0b995704ac None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:45.763423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3233f4-8815-4616-a8ef-ef0b995704ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:45.764432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3233f4-8815-4616-a8ef-ef0b995704ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:45.766561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3233f4-8815-4616-a8ef-ef0b995704ac None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:45.772000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:45.806947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2898e-058b-43b3-934e-3920e9f03068 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:45.822992 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:45.843989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 406/827] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:17:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:45.853160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:45.853160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:45.853160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:45.853160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:45.873847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:45.875817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:45.896601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:45.897593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:45.898501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:45.898501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:45.898501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:45.898501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:45.901047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 422/828] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:45.908175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3233f4-8815-4616-a8ef-ef0b995704ac None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:46.022358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3233f4-8815-4616-a8ef-ef0b995704ac None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:46.057013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:46.057694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:46.057974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:46.058245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:46.059312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:46.059617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:46.060980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:46.073572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 407/829] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:46.079248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:46.186777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a27074-bd8b-456f-9f2a-f0958c78926c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:46.222838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:46.223916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:46.224378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:46.224831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:46.232246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:46.233999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:46.248636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:46.249630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:46.250526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:46.250526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:46.250526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:46.250526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:46.250526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:46.250526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:46.250526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:46.250526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:46.252653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 423/830] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:46.258186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:46.316065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:46.364981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590cb2b-b259-4f73-b408-d75924a92c08 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:46.403300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a2a71-17c3-4b8c-aada-a6580042f3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:46.404154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a2a71-17c3-4b8c-aada-a6580042f3de None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:46.404539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a2a71-17c3-4b8c-aada-a6580042f3de None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:46.404914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a2a71-17c3-4b8c-aada-a6580042f3de None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:46.405840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a2a71-17c3-4b8c-aada-a6580042f3de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:46.406304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a2a71-17c3-4b8c-aada-a6580042f3de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:46.408121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a2a71-17c3-4b8c-aada-a6580042f3de None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:46.421741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 408/831] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:46.427526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:46.461454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:46.540267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3630db5-ed2e-4277-bd22-178227faeb1c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:46.575283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d601513-2130-48c6-a177-3423916fff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:46.575980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d601513-2130-48c6-a177-3423916fff20 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:46.576267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d601513-2130-48c6-a177-3423916fff20 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:46.576543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d601513-2130-48c6-a177-3423916fff20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:46.577393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d601513-2130-48c6-a177-3423916fff20 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f7ebbd8eb36c4919aedd3498bd8bc483) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:46.577711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d601513-2130-48c6-a177-3423916fff20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:46.579126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d601513-2130-48c6-a177-3423916fff20 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:46.585545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 424/832] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:46 2022] GET /identity/v3/role_assignments?user.id=f7ebbd8eb36c4919aedd3498bd8bc483&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=bb0756f8af694633b9743a958eb46e7a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:46.866815 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612bb7-e850-414b-994d-85447617dc25 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:46.867226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612bb7-e850-414b-994d-85447617dc25 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:46.867410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55612bb7-e850-414b-994d-85447617dc25 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:46.868983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 409/833] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:46.874944 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c28fc-b0eb-4645-86d7-7da7ad2fd429 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:46.875236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c28fc-b0eb-4645-86d7-7da7ad2fd429 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:46.875513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c28fc-b0eb-4645-86d7-7da7ad2fd429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:46.892371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-790c28fc-b0eb-4645-86d7-7da7ad2fd429 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:46.893585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-790c28fc-b0eb-4645-86d7-7da7ad2fd429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:46.971761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 425/834] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:17:46.996240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55612bb7-e850-414b-994d-85447617dc25 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:47.105167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55612bb7-e850-414b-994d-85447617dc25 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:47.120467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:47.141616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e55f1-ba90-418a-b7a2-4d7dc54ea778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:47.142441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e55f1-ba90-418a-b7a2-4d7dc54ea778 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:47.142727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e55f1-ba90-418a-b7a2-4d7dc54ea778 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:47.143006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e55f1-ba90-418a-b7a2-4d7dc54ea778 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:47.154342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e55f1-ba90-418a-b7a2-4d7dc54ea778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-ff8e55f1-ba90-418a-b7a2-4d7dc54ea778 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:47.162199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:47.163145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:47.164401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 410/835] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:47.170194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c28fc-b0eb-4645-86d7-7da7ad2fd429 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:47.187785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:47.276155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c28fc-b0eb-4645-86d7-7da7ad2fd429 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:47.311715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:47.312734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:47.313307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:47.313784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:47.314710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:47.315247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:47.316788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:47.326246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 426/836] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:17:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:47.333155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e55f1-ba90-418a-b7a2-4d7dc54ea778 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:47.440484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e55f1-ba90-418a-b7a2-4d7dc54ea778 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:47.476057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079cc39-53aa-401d-9b7e-17b07528bd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:47.476851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079cc39-53aa-401d-9b7e-17b07528bd99 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:47.477158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079cc39-53aa-401d-9b7e-17b07528bd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:47.477442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079cc39-53aa-401d-9b7e-17b07528bd99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:47.488390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079cc39-53aa-401d-9b7e-17b07528bd99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:47.489884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079cc39-53aa-401d-9b7e-17b07528bd99 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0079cc39-53aa-401d-9b7e-17b07528bd99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:47.507229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:47.508144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:47.509475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:47.509475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:47.509475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:47.509475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:47.509475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 411/837] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:47.515012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:47.563320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b521871-770c-45bc-a71d-b9feafecd85f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:47.563591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b521871-770c-45bc-a71d-b9feafecd85f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:47.563880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b521871-770c-45bc-a71d-b9feafecd85f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:47.565536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 412/838] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:47.570518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad12f-8c73-4434-a882-f805039230eb None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:47.570810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad12f-8c73-4434-a882-f805039230eb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:47.571101 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ad12f-8c73-4434-a882-f805039230eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:47.589373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5e1ad12f-8c73-4434-a882-f805039230eb None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:47.591304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5e1ad12f-8c73-4434-a882-f805039230eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:47.606938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:47.614884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:47.631337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db24e1f3-d128-4959-a394-f7198cdc2c1a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:47.667540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:47.668420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:47.668538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:47.668910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:47.669091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 413/839] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:17:47.669580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:47.669892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:47.671264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:47.689397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 427/840] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:47.696802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ad12f-8c73-4434-a882-f805039230eb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:47.698126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:47.811007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ad12f-8c73-4434-a882-f805039230eb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:47.814468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e201aa64-e717-4f7d-96f8-f8a91530feed None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:47.856745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:47.857779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:47.858187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:47.858528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:47.860520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c1230-3b15-450e-9207-e8163bdd1e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:47.861732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c1230-3b15-450e-9207-e8163bdd1e06 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:47.862160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c1230-3b15-450e-9207-e8163bdd1e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:47.862650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c1230-3b15-450e-9207-e8163bdd1e06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:47.867605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:47.869689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:47.871150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:47.875491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c1230-3b15-450e-9207-e8163bdd1e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a20c1230-3b15-450e-9207-e8163bdd1e06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:47.883884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:47.889608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:17:47.890711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:47.891331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:47.892507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 03:17:47.893957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:47.895023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 428/841] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:17:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:47.895148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 414/842] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:17:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:47.895283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c1230-3b15-450e-9207-e8163bdd1e06 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:47.895399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:47.918249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.007969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c1230-3b15-450e-9207-e8163bdd1e06 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.017351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c6ae0-3836-460e-8a4d-3900d90def27 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.043229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.044458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.044740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.045016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.045796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:48.046103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.047470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.060301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 429/843] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:17:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:48.061682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.064344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.065023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.065340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.065809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.067523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:48.067850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.069416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.097024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 415/844] 10.222.0.41 () {56 vars in 1135 bytes} [Sat May 28 03:17:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:48.098479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.140092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.180309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.184486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0c37c-5984-4c6c-a131-64dd36b6094b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.212275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a2ff1-7897-4b91-a95b-b9e247ec7d62 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.223253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f83fca-0442-4209-b37c-ce7ee1615855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.224043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f83fca-0442-4209-b37c-ce7ee1615855 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.224322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f83fca-0442-4209-b37c-ce7ee1615855 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.224593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f83fca-0442-4209-b37c-ce7ee1615855 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.236069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f83fca-0442-4209-b37c-ce7ee1615855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.237569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f83fca-0442-4209-b37c-ce7ee1615855 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-20f83fca-0442-4209-b37c-ce7ee1615855 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:48.254924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:48.255841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:48.256748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:48.256748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:48.256748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:48.256748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:48.256976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e9c81-254e-4eee-9f44-a1ea44ecb6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.256976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9c81-254e-4eee-9f44-a1ea44ecb6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.256976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9c81-254e-4eee-9f44-a1ea44ecb6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.257212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 430/845] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:48.257806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10334b68-8bfc-433b-a36c-84384f50891f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.258116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10334b68-8bfc-433b-a36c-84384f50891f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.258350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10334b68-8bfc-433b-a36c-84384f50891f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.261146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 431/846] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:48.261290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e9c81-254e-4eee-9f44-a1ea44ecb6a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.261384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10334b68-8bfc-433b-a36c-84384f50891f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.339995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e9c81-254e-4eee-9f44-a1ea44ecb6a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.342080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e9c81-254e-4eee-9f44-a1ea44ecb6a0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.377373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10334b68-8bfc-433b-a36c-84384f50891f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.391541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.413564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.414547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f0be0-0bc3-4274-9dbf-72cabf42c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.415218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0be0-0bc3-4274-9dbf-72cabf42c1ed None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.415513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0be0-0bc3-4274-9dbf-72cabf42c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.415794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f0be0-0bc3-4274-9dbf-72cabf42c1ed None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.416516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0be0-0bc3-4274-9dbf-72cabf42c1ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:48.416831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0be0-0bc3-4274-9dbf-72cabf42c1ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.418243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f0be0-0bc3-4274-9dbf-72cabf42c1ed None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.440987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 432/847] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:48.440987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba335270-222f-475c-90d7-8d73c495f43b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.440987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba335270-222f-475c-90d7-8d73c495f43b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.440987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba335270-222f-475c-90d7-8d73c495f43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.443982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 416/848] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:17:48.448748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e9c81-254e-4eee-9f44-a1ea44ecb6a0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.460837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ba335270-222f-475c-90d7-8d73c495f43b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:48.462073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ba335270-222f-475c-90d7-8d73c495f43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:48.536012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 433/849] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:48.541650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba335270-222f-475c-90d7-8d73c495f43b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.567277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e9c81-254e-4eee-9f44-a1ea44ecb6a0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.603911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.605172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.605929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.606624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.616173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.617658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:48.633857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:48.634849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:48.635813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:48.640332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 417/850] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:48.640931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.654122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba335270-222f-475c-90d7-8d73c495f43b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.656034 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.673222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.694042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.694829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.695112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.695387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.706808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.708243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:48.725841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:48.726984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:48.727943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:48.727943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:48.727943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:48.727943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:48.727943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 434/851] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:48.728224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.754558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42321457-9a75-4dbd-ba7f-43731fa382bc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.794822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.795567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.795853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.796127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.796790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.798242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:48.811544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:48.812880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:48.819088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 418/852] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:48.820055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.846766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74dd35-dcbc-4f0c-9b58-9d84fe18bbb0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.884639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.885507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.885798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.886086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.886917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:48.887295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.888725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.909904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 435/853] 10.222.0.41 () {56 vars in 1149 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:48.910560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:48.922948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.927912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:48.929827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71578074-320b-440c-9bb2-9ac6d7acca10 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:48.968301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:48.968972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:48.969291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:48.969572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:48.970296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:48.970606 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:48.972047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:48.984593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 419/854] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:48.985418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.016736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97fd76-cb68-4ba7-9208-1b45f5699990 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.050294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440551c-b015-4464-a4b9-3010a6217afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.050893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440551c-b015-4464-a4b9-3010a6217afa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.051187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440551c-b015-4464-a4b9-3010a6217afa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.051466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440551c-b015-4464-a4b9-3010a6217afa None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.052168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440551c-b015-4464-a4b9-3010a6217afa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:49.052477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440551c-b015-4464-a4b9-3010a6217afa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.053868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440551c-b015-4464-a4b9-3010a6217afa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.058291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 436/855] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:49.060055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b818c-5858-420f-8156-7e76bd5406d9 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.060351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b818c-5858-420f-8156-7e76bd5406d9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.060632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b818c-5858-420f-8156-7e76bd5406d9 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.061877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 437/856] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:49.062569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b818c-5858-420f-8156-7e76bd5406d9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.084607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cc5b5-f33b-4e76-912a-4765cba49085 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.118251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed426fae-f834-4c4e-9c18-3bb8e128195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.118893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed426fae-f834-4c4e-9c18-3bb8e128195a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.119173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed426fae-f834-4c4e-9c18-3bb8e128195a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.119446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed426fae-f834-4c4e-9c18-3bb8e128195a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.142900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:49.143729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:49.165729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b818c-5858-420f-8156-7e76bd5406d9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.186080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed426fae-f834-4c4e-9c18-3bb8e128195a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.187571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed426fae-f834-4c4e-9c18-3bb8e128195a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.201230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6c634-8b4a-47cd-8d6d-b5c204fa4669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.202131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6c634-8b4a-47cd-8d6d-b5c204fa4669 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.202419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6c634-8b4a-47cd-8d6d-b5c204fa4669 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.202698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6c634-8b4a-47cd-8d6d-b5c204fa4669 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.209441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6c634-8b4a-47cd-8d6d-b5c204fa4669 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.210866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6c634-8b4a-47cd-8d6d-b5c204fa4669 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-adc6c634-8b4a-47cd-8d6d-b5c204fa4669 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:49.225357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:49.226671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 438/857] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:49.227841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202377fa-07d0-4ac8-8f69-89c5215478d8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.228374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202377fa-07d0-4ac8-8f69-89c5215478d8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.228374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202377fa-07d0-4ac8-8f69-89c5215478d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.244480 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-202377fa-07d0-4ac8-8f69-89c5215478d8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:49.245689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-202377fa-07d0-4ac8-8f69-89c5215478d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:49.288633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 420/858] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:17:49.289479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed426fae-f834-4c4e-9c18-3bb8e128195a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.311800 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 439/859] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:49.312560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202377fa-07d0-4ac8-8f69-89c5215478d8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.345002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:49.387708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed426fae-f834-4c4e-9c18-3bb8e128195a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.391515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:49.421488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.422222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.422574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.422915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.423691 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:49.424062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.424859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202377fa-07d0-4ac8-8f69-89c5215478d8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.425531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.429918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 421/860] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:49.430681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.472744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.473456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.473743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.474022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.474825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:49.475136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.476605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.490185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 440/861] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:49.496585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.556118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ef0ae-d5c0-483d-8b87-be18230f652d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.578511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:49.594098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.594905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.595187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.598623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.607632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.609616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.625312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c87fae-6431-4292-b125-4f286e32724b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:49.627935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:49.629024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:49.629989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:49.629989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:49.629989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:49.629989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:49.629989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 422/862] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:49.629989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.662823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:49.673137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.684831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.684831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.684831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.684831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.684831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:49.690630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:49.691794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:49.697864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 441/863] 10.222.0.41 () {56 vars in 1132 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:49.698988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.747770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c5dbc-b45e-41c0-a3d3-0dacacd68e5e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.795565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.796221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.796509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.796777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.798301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f2f14380f60f4fa18ed6a9579cfb071b) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:49.798616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.800051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.806745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 423/864] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/role_assignments?user.id=f2f14380f60f4fa18ed6a9579cfb071b&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=bb0756f8af694633b9743a958eb46e7a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:49.807439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.846227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dafae4-0a4b-493b-bd6f-89dd4a1c03a5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.848664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:49.899446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.900693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.901178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.902177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.906740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:49.906740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.915560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.928177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 442/865] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:49.937176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2da8c-5668-444a-af9c-399fe20db0cc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:49.937385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:49.983368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:49.984700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:49.985242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:49.985751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:49.987931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:49.988603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:49.990290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:49.998098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 424/866] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:50.003419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:50.008142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:50.064976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4fa81-9ecb-4c50-add7-f6ae4e4f388b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:50.104692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:50.106474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:50.106474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:50.106645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:50.115170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:50.116629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:50.127400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:50.130884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64318b-777c-4514-b172-f954199cfc53 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:50.137499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:50.141444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:50.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 443/867] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:50.145827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:50.180550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:50.181257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:50.181958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:50.182259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:50.183128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:50.183562 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:50.185297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:50.192209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 425/868] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:17:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:50.273463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90ad81-faf4-4c52-91b2-be10ad809ee7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:50.286994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:50.313748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900b185-8685-465d-aa74-5c3d5f801be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:50.314935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900b185-8685-465d-aa74-5c3d5f801be4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:50.315427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900b185-8685-465d-aa74-5c3d5f801be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:50.315902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900b185-8685-465d-aa74-5c3d5f801be4 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:50.316933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900b185-8685-465d-aa74-5c3d5f801be4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:50.317464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900b185-8685-465d-aa74-5c3d5f801be4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:50.319239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900b185-8685-465d-aa74-5c3d5f801be4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:50.334311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 444/869] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:50.345180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:50.456818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa3d9-9243-49bb-a395-916990eb8ca3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:50.491779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05408dbf-e9c2-419f-b2a0-3b6115674e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:50.492434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05408dbf-e9c2-419f-b2a0-3b6115674e56 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:50.492715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05408dbf-e9c2-419f-b2a0-3b6115674e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:50.492992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05408dbf-e9c2-419f-b2a0-3b6115674e56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:50.493836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05408dbf-e9c2-419f-b2a0-3b6115674e56 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=e619c4b1fa304a8a99de2f60d1b6cc89) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:50.494148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05408dbf-e9c2-419f-b2a0-3b6115674e56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:50.495801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05408dbf-e9c2-419f-b2a0-3b6115674e56 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:50.502510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 426/870] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:50 2022] GET /identity/v3/role_assignments?user.id=e619c4b1fa304a8a99de2f60d1b6cc89&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:51.686037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10af479-b086-4910-bad8-b7ca28085e25 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:51.686555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10af479-b086-4910-bad8-b7ca28085e25 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:51.686665 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10af479-b086-4910-bad8-b7ca28085e25 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:51.688121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 445/871] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:51.693688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9407eef-bc36-4c38-a3f1-04d036d87801 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:51.694499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9407eef-bc36-4c38-a3f1-04d036d87801 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:51.694985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9407eef-bc36-4c38-a3f1-04d036d87801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:51.712938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b9407eef-bc36-4c38-a3f1-04d036d87801 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:51.714565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b9407eef-bc36-4c38-a3f1-04d036d87801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:51.740365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:51.784810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 427/872] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:17:51.807267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10af479-b086-4910-bad8-b7ca28085e25 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:51.921627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10af479-b086-4910-bad8-b7ca28085e25 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:51.961353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a56fa3-509a-47ea-b7d8-21a2bc46091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:51.961943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a56fa3-509a-47ea-b7d8-21a2bc46091f None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:51.962228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a56fa3-509a-47ea-b7d8-21a2bc46091f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:51.962507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a56fa3-509a-47ea-b7d8-21a2bc46091f None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:51.963231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a56fa3-509a-47ea-b7d8-21a2bc46091f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:51.964606 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a56fa3-509a-47ea-b7d8-21a2bc46091f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:52.041580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 446/873] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:17:51 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:52.342920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ccb9f-7254-404a-b031-f369c7a19fca None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.343305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ccb9f-7254-404a-b031-f369c7a19fca None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.343510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ccb9f-7254-404a-b031-f369c7a19fca None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.345090 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 428/874] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:52.351037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c41f8-bb64-450a-87b3-2c3a87be9780 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.351336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c41f8-bb64-450a-87b3-2c3a87be9780 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.351626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c41f8-bb64-450a-87b3-2c3a87be9780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.368754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fe6c41f8-bb64-450a-87b3-2c3a87be9780 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:52.369974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fe6c41f8-bb64-450a-87b3-2c3a87be9780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:52.396267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:52.452087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 447/875] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:52.475771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ccb9f-7254-404a-b031-f369c7a19fca None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:52.521783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f8b87-f4b1-47b9-bf96-6fd6609d7732 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.522088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f8b87-f4b1-47b9-bf96-6fd6609d7732 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.522371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f8b87-f4b1-47b9-bf96-6fd6609d7732 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.523988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 448/876] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:52.529529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b05b-339b-46a8-80a2-5b06b3f9d33b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.529816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b05b-339b-46a8-80a2-5b06b3f9d33b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.530095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9b05b-339b-46a8-80a2-5b06b3f9d33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.551465 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bef9b05b-339b-46a8-80a2-5b06b3f9d33b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:52.552798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bef9b05b-339b-46a8-80a2-5b06b3f9d33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:52.589925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ccb9f-7254-404a-b031-f369c7a19fca None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:52.637938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:52.638710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.638941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.639202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.639963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:52.641363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:52.649510 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 449/877] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:52.654546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:52.655558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:52.656604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 429/878] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:17:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:52.663934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b05b-339b-46a8-80a2-5b06b3f9d33b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:52.669999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:52.706632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:52.761261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:52.780317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9b05b-339b-46a8-80a2-5b06b3f9d33b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:52.800536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acecc1-adba-4024-a382-6bb01669ba99 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:52.827450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:52.828227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.828515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.828792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.829784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:52.830102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:52.832157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:52.838235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 450/879] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:17:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:52.839277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3b2e9-65e8-4cc4-a23e-c94bbead80f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:52.840021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b2e9-65e8-4cc4-a23e-c94bbead80f5 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.840419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b2e9-65e8-4cc4-a23e-c94bbead80f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.848935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3b2e9-65e8-4cc4-a23e-c94bbead80f5 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.848935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3b2e9-65e8-4cc4-a23e-c94bbead80f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:52.848935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a3b2e9-65e8-4cc4-a23e-c94bbead80f5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:52.849277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:52.943553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 430/880] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:17:52 2022] POST /identity/v3/services => generated 269 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:17:52.945016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9055-8db4-4346-9c27-cd281f7dfe4e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.945339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9055-8db4-4346-9c27-cd281f7dfe4e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.945615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a9055-8db4-4346-9c27-cd281f7dfe4e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.946985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 431/881] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:52.952895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80652b-cc8e-49a7-83f4-b63c55ea7d5e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:52.953561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80652b-cc8e-49a7-83f4-b63c55ea7d5e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:52.954070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80652b-cc8e-49a7-83f4-b63c55ea7d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:52.961743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0229b9-9f2b-4572-909c-697d2b342556 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:52.973875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9d80652b-cc8e-49a7-83f4-b63c55ea7d5e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:52.975476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9d80652b-cc8e-49a7-83f4-b63c55ea7d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:53.019494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:53.020189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:53.020540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:53.020742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:53.021535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:53.021848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:53.023454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:53.036091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 451/882] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:17:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:53.074418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 432/883] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:17:53.099700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:53.115297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:53.220419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0397b-b645-4c42-b9ee-e9771a284f07 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:53.258584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4cadf-a8ed-45f2-a6ad-426048b310d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:53.259702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4cadf-a8ed-45f2-a6ad-426048b310d6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:53.260185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4cadf-a8ed-45f2-a6ad-426048b310d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:53.260614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4cadf-a8ed-45f2-a6ad-426048b310d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:53.271779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4cadf-a8ed-45f2-a6ad-426048b310d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-33c4cadf-a8ed-45f2-a6ad-426048b310d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:53.280928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:53.282134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:17:53.283214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:53.285653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 452/884] 10.222.0.41 () {56 vars in 1120 bytes} [Sat May 28 03:17:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:53.291705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80652b-cc8e-49a7-83f4-b63c55ea7d5e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:53.307510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:53.401487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80652b-cc8e-49a7-83f4-b63c55ea7d5e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:53.437902 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:53.438925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:53.439427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:53.439909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:53.440851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:53.441432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:53.443187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:53.452957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 433/885] 10.222.0.41 () {56 vars in 1115 bytes} [Sat May 28 03:17:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:53.458166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4cadf-a8ed-45f2-a6ad-426048b310d6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:53.531613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:53.563764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4cadf-a8ed-45f2-a6ad-426048b310d6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:53.600463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:53.601293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:53.601583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:53.601859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:53.613096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:53.614629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:53.640024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:53.641306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:53.642345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:53.642345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:53.642345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:53.642345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:53.642345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 453/886] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:53.654411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:53.738269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:53.769853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf692ed-84ce-4841-9248-0d3f85d07bc2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:53.806186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:53.806776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:53.807062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:53.807343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:53.808070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:53.808379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:53.810329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:53.823055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 434/887] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:53.829248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:53.936305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13131-767f-49cd-80f7-2c65d155022a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:53.960247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:53.971920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:53.972995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:53.973374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:53.973675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:53.981421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:53.986261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:53.997932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:53.998863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:53.999814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 454/888] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:17:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:54.006630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:54.115059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4e31-1646-47bf-ae58-c294b2b4d6c3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:54.155194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d3ef9-4098-4e4a-b152-a50eede3bd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:54.155928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d3ef9-4098-4e4a-b152-a50eede3bd21 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.156222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d3ef9-4098-4e4a-b152-a50eede3bd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.156502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d3ef9-4098-4e4a-b152-a50eede3bd21 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.157376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d3ef9-4098-4e4a-b152-a50eede3bd21 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:54.157702 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d3ef9-4098-4e4a-b152-a50eede3bd21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:54.159145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d3ef9-4098-4e4a-b152-a50eede3bd21 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:54.160954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:54.178130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 435/889] 10.222.0.41 () {56 vars in 1149 bytes} [Sat May 28 03:17:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:54.183858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:54.295033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ea763-8f39-4f20-afb6-35f9c51bdfac None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:54.318599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401c0b-ecb3-4a7e-9039-3b697750db68 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.318810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401c0b-ecb3-4a7e-9039-3b697750db68 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.319061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401c0b-ecb3-4a7e-9039-3b697750db68 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.320671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 436/890] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:54.325809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ced397-88b1-456f-af4b-5fa9d92dd423 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.326037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ced397-88b1-456f-af4b-5fa9d92dd423 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.326264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ced397-88b1-456f-af4b-5fa9d92dd423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.341674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eb2b2-8a9c-4ee5-9692-2587faff3e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:54.343376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eb2b2-8a9c-4ee5-9692-2587faff3e28 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.344045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eb2b2-8a9c-4ee5-9692-2587faff3e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.344690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1eb2b2-8a9c-4ee5-9692-2587faff3e28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.346735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-72ced397-88b1-456f-af4b-5fa9d92dd423 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:54.348194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-72ced397-88b1-456f-af4b-5fa9d92dd423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:54.409049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:54.429618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 437/891] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:54.434719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eb2b2-8a9c-4ee5-9692-2587faff3e28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:54.434719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1eb2b2-8a9c-4ee5-9692-2587faff3e28 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:54.454986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ced397-88b1-456f-af4b-5fa9d92dd423 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:54.544671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 455/892] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:17:54 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:17:54.549840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eb2b2-8a9c-4ee5-9692-2587faff3e28 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:54.575470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ced397-88b1-456f-af4b-5fa9d92dd423 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:54.599969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:54.613283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:54.614404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.615008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.615552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.616463 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:54.618235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:54.631659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:54.632974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:54.633848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:54.641633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 438/893] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:54.645404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:54.647973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:54.672137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1eb2b2-8a9c-4ee5-9692-2587faff3e28 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:54.716311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:54.717547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.718057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.718592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.739809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:54.741761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:54.761289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:54.762262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:54.763174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:54.763174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:54.763174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:54.763174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:54.765599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 456/894] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:54.770888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:54.772966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd90b2c-31d2-4198-8f23-e722785eafa0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:54.815703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:54.816375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.816661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.816940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.818205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:54.818523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:54.819921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:54.831709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 439/895] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:54.832578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:54.884107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:54.890783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e23a54-00cd-4214-a348-d6751f629a89 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:54.929793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:54.930579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.931013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.931393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.932934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:54.933396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:54.935730 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:54.937519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:54.945228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ffd81-01a8-43a6-aff8-f6c70eb09a3f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:54.954569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 457/896] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:54.959080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:54.994037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:54.995075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:54.995432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:54.995784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:54.996873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:54.997279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:54.999609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:55.011392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 440/897] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:55.013355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:55.087413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c10b4c-d037-433d-85d7-f7f189b71975 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:55.131494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:55.132743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.133407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.133708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.144376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:55.145236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbef6ea-10f9-4ac9-86b7-0e43f9cfd8f0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:55.146403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:55.161126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:55.165772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:55.166672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:55.167568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:55.167568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:55.167568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:55.167568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:55.167568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:55.167568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:55.167568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:55.167568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:55.174847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 458/898] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:55.176538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:55.190828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:55.191494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.191779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.192057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.193007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] RBAC: Adding query filter params (interface=public, service_id=64c9abc5acf3450a927f7a8f3445aa7e, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:55.193974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:55.196051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:55.209299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 441/899] 10.222.0.41 () {56 vars in 1251 bytes} [Sat May 28 03:17:55 2022] GET /identity/v3/endpoints?service_id=64c9abc5acf3450a927f7a8f3445aa7e&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:55.211169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:55.253899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:55.301554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd05745-2b5a-419f-b3a0-4336c34a2f13 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:55.326522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51152f-cf04-42d3-9114-326094a48a36 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:55.339414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:55.340100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.340390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.340669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.341487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:55.341798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:55.343343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:55.361476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 459/900] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:55.363115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:55.373361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274fb33-991f-4817-b9b4-5fdf514d42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:55.374031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274fb33-991f-4817-b9b4-5fdf514d42f7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.374315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274fb33-991f-4817-b9b4-5fdf514d42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.374591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274fb33-991f-4817-b9b4-5fdf514d42f7 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.375278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274fb33-991f-4817-b9b4-5fdf514d42f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:55.376657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274fb33-991f-4817-b9b4-5fdf514d42f7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:55.393135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 442/901] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:17:55 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:17:55.395270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab9c3e-2385-4c55-be72-cae30e7c736b None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.395567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab9c3e-2385-4c55-be72-cae30e7c736b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.395848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab9c3e-2385-4c55-be72-cae30e7c736b None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.397428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 443/902] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:55.398490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dd197-5d64-4a25-a133-e416484865f6 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.398779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dd197-5d64-4a25-a133-e416484865f6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.399123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dd197-5d64-4a25-a133-e416484865f6 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.400662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 444/903] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:55.403847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fedb3-4a7a-4808-b347-feb3d6e66efa None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.404156 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fedb3-4a7a-4808-b347-feb3d6e66efa None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.404406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fedb3-4a7a-4808-b347-feb3d6e66efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.421881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8c6fedb3-4a7a-4808-b347-feb3d6e66efa None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:55.423107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8c6fedb3-4a7a-4808-b347-feb3d6e66efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:55.467776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:55.494901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82524bf6-d4ce-46b4-b4cb-fd889e3479cb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:55.501727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 445/904] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:17:55.503054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801813b4-fb44-499e-8751-f1ebd71ebb29 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.503350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801813b4-fb44-499e-8751-f1ebd71ebb29 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.503627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801813b4-fb44-499e-8751-f1ebd71ebb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.519153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:55.523453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-801813b4-fb44-499e-8751-f1ebd71ebb29 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:55.524901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-801813b4-fb44-499e-8751-f1ebd71ebb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:55.545409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:55.546404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.546853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.547288 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.548808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] RBAC: Adding query filter params (role.id=83d48564c08d484481c9ec8706230ee6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=e619c4b1fa304a8a99de2f60d1b6cc89) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:55.549329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:55.551390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:55.561981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 460/905] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:17:55 2022] GET /identity/v3/role_assignments?user.id=e619c4b1fa304a8a99de2f60d1b6cc89&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=83d48564c08d484481c9ec8706230ee6 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:55.563047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:55.605435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 446/906] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:55.630547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801813b4-fb44-499e-8751-f1ebd71ebb29 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:55.717947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fdbbf-65e2-4627-8369-4e66c03f5e40 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:55.744986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801813b4-fb44-499e-8751-f1ebd71ebb29 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:55.747605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:55.766035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f25162-0ae5-44bf-a18f-14ad466905a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:55.776133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25162-0ae5-44bf-a18f-14ad466905a7 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.776133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25162-0ae5-44bf-a18f-14ad466905a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.776133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25162-0ae5-44bf-a18f-14ad466905a7 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.776133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f25162-0ae5-44bf-a18f-14ad466905a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:55.776133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f25162-0ae5-44bf-a18f-14ad466905a7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:55.784780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:55.785639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:55.785928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:55.786203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:55.786964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:55.788341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:55.802005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:55.810425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:55.811240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 447/907] 10.222.0.41 () {56 vars in 1132 bytes} [Sat May 28 03:17:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:55.812138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:55.865168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 461/908] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:17:55 2022] POST /identity/v3/services => generated 241 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:55.947950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85df9a9-34e3-4059-9a29-6f5678fecf1c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:56.009147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:56.010115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:56.010577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:56.011019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:56.011938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:56.012406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:56.014070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:56.020581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 448/909] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:17:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:56.026666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f25162-0ae5-44bf-a18f-14ad466905a7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:56.074300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:56.137530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f25162-0ae5-44bf-a18f-14ad466905a7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:56.173973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:56.174701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:56.174996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:56.175277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:56.175994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:56.176315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:56.177732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:56.183324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 462/910] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:17:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:56.189042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:56.218843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:56.295398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387f010-b9d5-466a-b9d4-d3f67125b54a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:56.331617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805444cc-5dd7-4f73-94f3-49ef1ec952f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:56.332280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805444cc-5dd7-4f73-94f3-49ef1ec952f3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:56.332567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805444cc-5dd7-4f73-94f3-49ef1ec952f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:56.332848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805444cc-5dd7-4f73-94f3-49ef1ec952f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:56.333670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805444cc-5dd7-4f73-94f3-49ef1ec952f3 None admin] RBAC: Adding query filter params (interface=public, service_id=2cd2e680160a456d959548c87dad2444, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:56.333987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805444cc-5dd7-4f73-94f3-49ef1ec952f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:56.335363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805444cc-5dd7-4f73-94f3-49ef1ec952f3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:56.345458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 449/911] 10.222.0.41 () {56 vars in 1251 bytes} [Sat May 28 03:17:56 2022] GET /identity/v3/endpoints?service_id=2cd2e680160a456d959548c87dad2444&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:56.347023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:56.451490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138535df-50b8-4680-9e06-8d9234badb93 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:56.453380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:56.488758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7f73b-b33f-441e-a860-eaded08a1196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:56.489815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7f73b-b33f-441e-a860-eaded08a1196 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:56.490318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7f73b-b33f-441e-a860-eaded08a1196 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:56.490796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7f73b-b33f-441e-a860-eaded08a1196 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:56.491675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7f73b-b33f-441e-a860-eaded08a1196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:56.494005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7f73b-b33f-441e-a860-eaded08a1196 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:56.500747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 463/912] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:17:56 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:17:57.661935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e57380-f460-457d-be5f-9c494e8eefae None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:57.662497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e57380-f460-457d-be5f-9c494e8eefae None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:57.662497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e57380-f460-457d-be5f-9c494e8eefae None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:57.663972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 450/913] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:57.670015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3064bc-6e65-4529-a80a-5a223bbb05f1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:57.670860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3064bc-6e65-4529-a80a-5a223bbb05f1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:57.671344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3064bc-6e65-4529-a80a-5a223bbb05f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:57.688724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ce3064bc-6e65-4529-a80a-5a223bbb05f1 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:57.690271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ce3064bc-6e65-4529-a80a-5a223bbb05f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:57.765256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 464/914] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:57.784563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e57380-f460-457d-be5f-9c494e8eefae None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:57.822792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f51b2-62ce-42c5-8b7f-0a11b07d37da None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:57.823409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f51b2-62ce-42c5-8b7f-0a11b07d37da None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:57.823939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f51b2-62ce-42c5-8b7f-0a11b07d37da None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:57.826454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 465/915] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:17:57.832296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ce35-ea29-47d1-9933-acb815b58fb3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:57.832619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ce35-ea29-47d1-9933-acb815b58fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:57.832877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ce35-ea29-47d1-9933-acb815b58fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:57.854961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-60a7ce35-ea29-47d1-9933-acb815b58fb3 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:57.856776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-60a7ce35-ea29-47d1-9933-acb815b58fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:57.886396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:57.897992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e57380-f460-457d-be5f-9c494e8eefae None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:57.936847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:57.937963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:57.938503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:57.938991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:57.939805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:57.941387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:57.954424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:57.955503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 03:17:57.956328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:57.963709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 466/917] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:57.963971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 451/917] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:57.966503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7ce35-ea29-47d1-9933-acb815b58fb3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:57.991775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.002113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:58.089579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7ce35-ea29-47d1-9933-acb815b58fb3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.101659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb9962-1bcd-4eb5-a5b3-ac99a82c2781 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.125937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.126605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.126888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.127426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.128572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:58.128885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:58.130346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:58.135700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 467/918] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:58.139006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844d341-7429-4ec7-a576-b49e7d94fa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.140062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d341-7429-4ec7-a576-b49e7d94fa08 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.140697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d341-7429-4ec7-a576-b49e7d94fa08 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.141214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d341-7429-4ec7-a576-b49e7d94fa08 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.141797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.153090 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844d341-7429-4ec7-a576-b49e7d94fa08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0844d341-7429-4ec7-a576-b49e7d94fa08 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:17:58.161626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:58.162566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:17:58.163431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:58.170836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 452/919] 10.222.0.41 () {56 vars in 1120 bytes} [Sat May 28 03:17:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:58.172316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844d341-7429-4ec7-a576-b49e7d94fa08 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.189580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:58.239265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:58.250666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b8a7b-5522-403e-b354-baaca3431a88 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.278149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844d341-7429-4ec7-a576-b49e7d94fa08 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.287017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d05f1-69af-483c-94f8-ea5650b277bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.288042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d05f1-69af-483c-94f8-ea5650b277bf None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.288332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d05f1-69af-483c-94f8-ea5650b277bf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.288615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d05f1-69af-483c-94f8-ea5650b277bf None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.289655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d05f1-69af-483c-94f8-ea5650b277bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:58.289966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d05f1-69af-483c-94f8-ea5650b277bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:58.291338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d05f1-69af-483c-94f8-ea5650b277bf None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:58.300643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 468/920] 10.222.0.41 () {56 vars in 1114 bytes} [Sat May 28 03:17:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:58.300643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109585e-34c5-47af-b957-2782ea3b4d2f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.300643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109585e-34c5-47af-b957-2782ea3b4d2f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.300643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109585e-34c5-47af-b957-2782ea3b4d2f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.300643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 469/921] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:58.301552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109585e-34c5-47af-b957-2782ea3b4d2f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.317557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2598ce1-b7ec-49e6-b569-7189ce869bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.318223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598ce1-b7ec-49e6-b569-7189ce869bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.318504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598ce1-b7ec-49e6-b569-7189ce869bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.318779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2598ce1-b7ec-49e6-b569-7189ce869bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.319508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2598ce1-b7ec-49e6-b569-7189ce869bb4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:58.319815 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2598ce1-b7ec-49e6-b569-7189ce869bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:58.321213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2598ce1-b7ec-49e6-b569-7189ce869bb4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:58.334961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 453/922] 10.222.0.41 () {56 vars in 1115 bytes} [Sat May 28 03:17:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:58.336329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ce1ce-5b5b-417f-b80c-bc9416690fd2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.336621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ce1ce-5b5b-417f-b80c-bc9416690fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.336899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ce1ce-5b5b-417f-b80c-bc9416690fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.353219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-710ce1ce-5b5b-417f-b80c-bc9416690fd2 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:58.354389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-710ce1ce-5b5b-417f-b80c-bc9416690fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:58.409731 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109585e-34c5-47af-b957-2782ea3b4d2f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.420218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 454/923] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:58.425956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ce1ce-5b5b-417f-b80c-bc9416690fd2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.435929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:58.447243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebc96b-3d9e-4e53-8559-69ee9689ba29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.450442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebc96b-3d9e-4e53-8559-69ee9689ba29 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.450442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebc96b-3d9e-4e53-8559-69ee9689ba29 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.450442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebc96b-3d9e-4e53-8559-69ee9689ba29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.450442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebc96b-3d9e-4e53-8559-69ee9689ba29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:58.451292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdebc96b-3d9e-4e53-8559-69ee9689ba29 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:58.523460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 470/924] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:17:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:17:58.524618 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebc96b-3d9e-4e53-8559-69ee9689ba29 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.542430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ce1ce-5b5b-417f-b80c-bc9416690fd2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.546215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:58.612401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.613390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.613772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.614320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.629839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:58.631648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:58.656360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:58.657353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:58.658250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:58.658250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:58.658250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:58.658250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:58.658250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 455/925] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:58.663972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.672917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebc96b-3d9e-4e53-8559-69ee9689ba29 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.716510 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.717720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.718294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.718908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.719841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:58.722007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:58.725284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:58.737338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:58.746208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:58.747581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 471/926] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:17:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:58.748372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.783791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0c608-dca4-4227-87d2-2faaf71fc903 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.819266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99b2bf-5d35-4a07-a82e-40411c9cfa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.819927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99b2bf-5d35-4a07-a82e-40411c9cfa30 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.820207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99b2bf-5d35-4a07-a82e-40411c9cfa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.820478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99b2bf-5d35-4a07-a82e-40411c9cfa30 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.821208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99b2bf-5d35-4a07-a82e-40411c9cfa30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:58.821521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99b2bf-5d35-4a07-a82e-40411c9cfa30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:58.822886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99b2bf-5d35-4a07-a82e-40411c9cfa30 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:58.831275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:58.835190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 456/927] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:58.836279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba625bf1-b84b-463c-8baa-fead07d5561a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.836570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba625bf1-b84b-463c-8baa-fead07d5561a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.836849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba625bf1-b84b-463c-8baa-fead07d5561a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.842024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 457/928] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:17:58.842024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba625bf1-b84b-463c-8baa-fead07d5561a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:58.866516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b7870-237b-4649-9306-49a62c56fcee None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.910592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656b623-0329-434c-aab9-6eaa347be828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.915681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656b623-0329-434c-aab9-6eaa347be828 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.915681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656b623-0329-434c-aab9-6eaa347be828 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.915681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656b623-0329-434c-aab9-6eaa347be828 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.915681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656b623-0329-434c-aab9-6eaa347be828 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:58.916393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656b623-0329-434c-aab9-6eaa347be828 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:58.920724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656b623-0329-434c-aab9-6eaa347be828 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:58.926828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 472/929] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:17:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:17:58.930470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7c3a0-5b9c-4a28-982c-c0b91f09fc4d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.930819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7c3a0-5b9c-4a28-982c-c0b91f09fc4d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.931166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7c3a0-5b9c-4a28-982c-c0b91f09fc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:58.947751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ced7c3a0-5b9c-4a28-982c-c0b91f09fc4d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:17:58.948963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ced7c3a0-5b9c-4a28-982c-c0b91f09fc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:17:58.958743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba625bf1-b84b-463c-8baa-fead07d5561a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:58.974159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:58.995022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:58.996008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:58.999742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:58.999742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.004838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.006331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:17:59.020932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:59.021914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:59.022844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 458/930] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:17:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:59.024168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.041148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 473/931] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:17:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:17:59.041148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7c3a0-5b9c-4a28-982c-c0b91f09fc4d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.119781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:59.136824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d2777-2954-41f2-aed0-5c8a2ef09906 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.155585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7c3a0-5b9c-4a28-982c-c0b91f09fc4d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.173308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.173971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.174254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.174537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.175237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:59.175546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.176972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.183078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 459/932] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:59.183891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.197251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.198180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.198462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.198735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.199594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:59.199903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.201361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.219275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 474/933] 10.222.0.41 () {56 vars in 1149 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:59.220064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.234203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:59.298289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7139f6-0317-4f20-823e-2f2d59d837b7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.322835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22dc76-cf78-4dd5-8cda-c0a0d9baec67 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.333010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.333859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.334143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.334420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.335067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.336445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:59.349905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:59.350962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:59.351806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 460/934] 10.222.0.41 () {56 vars in 1132 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:59.352642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.358471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:59.369044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.369757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.370029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.370308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.371092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] RBAC: Adding query filter params (interface=public, service_id=b7e3ae5d60c24c9bbfb358438853b95b, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:59.371439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.373050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.385370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 475/935] 10.222.0.41 () {56 vars in 1251 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/endpoints?service_id=b7e3ae5d60c24c9bbfb358438853b95b&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:59.385370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.463169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a19b7d-d77a-4b96-a8eb-362f1a225397 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.464729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:59.492505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995d792-5468-4b7a-bb34-9abb303e673d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.502197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.503077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.503363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.503645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.514421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.515866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.529901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.530784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.531250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.531719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.532622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:59.533139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:17:59.533962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:59.534824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:59.535718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:17:59.535718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:17:59.535718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:17:59.535718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:17:59.536226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.539827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 461/936] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:17:59.540414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.545761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 476/937] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:59.545761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.600281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:59.647706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc989c-2fe4-4d50-a9db-a3406a17f93a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.649255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc9101-6355-462d-a155-8c1f52cf6914 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.683652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d046f5-096b-40e0-a5dc-c4a93442c56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.684980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d046f5-096b-40e0-a5dc-c4a93442c56f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.685381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.686040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.686159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d046f5-096b-40e0-a5dc-c4a93442c56f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.686317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.686592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.687064 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d046f5-096b-40e0-a5dc-c4a93442c56f None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.687381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:59.687692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.688387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d046f5-096b-40e0-a5dc-c4a93442c56f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.689097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.690260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d046f5-096b-40e0-a5dc-c4a93442c56f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.697905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 462/938] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:17:59.699131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d046f5-096b-40e0-a5dc-c4a93442c56f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.713893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 477/939] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:59.720129 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.732356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:59.848644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c00a9-76fe-4302-bb85-b8a2b5344164 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.850398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d046f5-096b-40e0-a5dc-c4a93442c56f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:17:59.865894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:17:59.892888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.893353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.893902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.894190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.908147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:17:59.908147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:17:59.909769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.909850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:17:59.909850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:17:59.909850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:17:59.910026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.910106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:17:59.911155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:17:59.920500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 463/940] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:17:59.927443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:17:59.929546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:17:59.930410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:17:59.931276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 478/941] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:17:59.938138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:00.020668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:00.041638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99610c68-8b50-405b-94d4-247bb98de7a9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:00.078389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b45b9-600f-4a72-8d8b-5d6bc65a6f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:00.079218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b45b9-600f-4a72-8d8b-5d6bc65a6f9e None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:00.079738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b45b9-600f-4a72-8d8b-5d6bc65a6f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:00.079972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b45b9-600f-4a72-8d8b-5d6bc65a6f9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:00.080839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b45b9-600f-4a72-8d8b-5d6bc65a6f9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:00.082438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b45b9-600f-4a72-8d8b-5d6bc65a6f9e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:00.083122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10a9e-e414-46d7-b7f7-91bacbaa6572 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:00.121928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:00.122583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:00.122861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:00.123132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:00.123909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:00.124215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:00.125657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:00.138411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 479/942] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:00.144033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:00.179152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:00.213625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 464/943] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:17:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:00.263852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073c6ca-7362-4bfb-9506-1e780a84f6ef None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:00.310456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784da205-c34e-450d-b024-2dbfc836e83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:00.311654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da205-c34e-450d-b024-2dbfc836e83f None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:00.312090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da205-c34e-450d-b024-2dbfc836e83f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:00.312520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784da205-c34e-450d-b024-2dbfc836e83f None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/e619c4b1fa304a8a99de2f60d1b6cc89/roles/83d48564c08d484481c9ec8706230ee6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:00.315144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784da205-c34e-450d-b024-2dbfc836e83f None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=e619c4b1fa304a8a99de2f60d1b6cc89, role_id=83d48564c08d484481c9ec8706230ee6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:00.316927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784da205-c34e-450d-b024-2dbfc836e83f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:00.380650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 480/944] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:18:00 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/e619c4b1fa304a8a99de2f60d1b6cc89/roles/83d48564c08d484481c9ec8706230ee6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:18:00.843551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee95cd1-7ece-416c-9fa5-f009d8cb96f5 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:00.844096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee95cd1-7ece-416c-9fa5-f009d8cb96f5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:00.844096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee95cd1-7ece-416c-9fa5-f009d8cb96f5 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:00.845778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 465/945] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:00.851440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eac169-e280-4edf-a897-c4439307b4f6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:00.851843 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eac169-e280-4edf-a897-c4439307b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:00.852013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eac169-e280-4edf-a897-c4439307b4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:00.869010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-97eac169-e280-4edf-a897-c4439307b4f6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:00.870278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-97eac169-e280-4edf-a897-c4439307b4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:00.956848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 481/946] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:00.980297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee95cd1-7ece-416c-9fa5-f009d8cb96f5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:01.091326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee95cd1-7ece-416c-9fa5-f009d8cb96f5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:01.130594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:01.131594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:01.132088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:01.132618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:01.133596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:01.135278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:01.136895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING keystone.server.flask.common [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 28 03:18:01.145824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:01.194146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.sql.core [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 03:18:01.194146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 03:18:01.194146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [parameters: {'id': '1c2b540450bc47e7909cc358dc9d4381', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 28 03:18:01.194146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80355) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1c2b540450bc47e7909cc358dc9d4381.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1c2b540450bc47e7909cc358dc9d4381. May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result = self._query(query) May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application conn.query(q) May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 03:18:01.228365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result.read() May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 28 03:18:01.229315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return project_ref.to_dict() May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application next(self.gen) May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application yield resource May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application next(self.gen) May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.session.rollback() May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.force_reraise() May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise self.value May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application session.commit() May 28 03:18:01.230293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._prepare_impl() May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.session.flush() May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._flush(objects) May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application compat.raise_( May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application flush_context.execute() May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 03:18:01.231767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rec.execute(self) May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application c = connection._execute_20( May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 03:18:01.232652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result = self._query(query) May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application conn.query(q) May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application result.read() May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 03:18:01.238814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [parameters: {'id': '1c2b540450bc47e7909cc358dc9d4381', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:01.240947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:01.242922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.Conflict( May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1c2b540450bc47e7909cc358dc9d4381. May 28 03:18:01.245398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:01.248161 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 466/947] 10.222.0.41 () {60 vars in 1140 bytes} [Sat May 28 03:18:00 2022] POST /identity/v3/projects => generated 262 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:01.258274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eac169-e280-4edf-a897-c4439307b4f6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:01.292305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:01.422861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eac169-e280-4edf-a897-c4439307b4f6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:01.496420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e6d22-bd0e-4894-ab18-e012ebee2b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:01.497257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e6d22-bd0e-4894-ab18-e012ebee2b4e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:01.499150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e6d22-bd0e-4894-ab18-e012ebee2b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:01.499449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e6d22-bd0e-4894-ab18-e012ebee2b4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:01.513966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e6d22-bd0e-4894-ab18-e012ebee2b4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:01.517724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e6d22-bd0e-4894-ab18-e012ebee2b4e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-041e6d22-bd0e-4894-ab18-e012ebee2b4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:18:01.553251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:01.554236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:01.555124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:18:01.555124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:18:01.555124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:18:01.555124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:18:01.555124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:18:01.555124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:18:01.555124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:01.555124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:01.559460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 482/948] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:18:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:01.562046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:01.715713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826511dc-03e1-440f-9f37-4e48c93366b0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:01.750747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87639725-0f60-4369-87ae-be1b16f956fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:01.751743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87639725-0f60-4369-87ae-be1b16f956fe None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:01.752252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87639725-0f60-4369-87ae-be1b16f956fe None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:01.752743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87639725-0f60-4369-87ae-be1b16f956fe None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:01.753723 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87639725-0f60-4369-87ae-be1b16f956fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:01.754259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87639725-0f60-4369-87ae-be1b16f956fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:01.756122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87639725-0f60-4369-87ae-be1b16f956fe None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:01.778819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 467/949] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:18:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:02.019806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25299-0d00-4f27-99bd-114fc3d9d65d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.020436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25299-0d00-4f27-99bd-114fc3d9d65d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.020436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c25299-0d00-4f27-99bd-114fc3d9d65d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.022130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 483/950] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:02.027967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11552188-5fec-4e9a-bc9a-9868caea321c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.028408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11552188-5fec-4e9a-bc9a-9868caea321c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.028596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11552188-5fec-4e9a-bc9a-9868caea321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.046148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-11552188-5fec-4e9a-bc9a-9868caea321c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:02.047658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-11552188-5fec-4e9a-bc9a-9868caea321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:02.099162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:02.120180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 468/951] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:02.142598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c25299-0d00-4f27-99bd-114fc3d9d65d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:02.230332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:02.251421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c25299-0d00-4f27-99bd-114fc3d9d65d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:02.288182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:02.288914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.289248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.289525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.291017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:02.292457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:02.310308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:02.311450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:02.312338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 484/952] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:18:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:02.317700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11552188-5fec-4e9a-bc9a-9868caea321c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:02.431489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11552188-5fec-4e9a-bc9a-9868caea321c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:02.485539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c8e76-6acf-4616-927d-fb5ff40dde6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:02.486525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8e76-6acf-4616-927d-fb5ff40dde6c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.486951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8e76-6acf-4616-927d-fb5ff40dde6c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.487366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c8e76-6acf-4616-927d-fb5ff40dde6c None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.488462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c8e76-6acf-4616-927d-fb5ff40dde6c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:02.488934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c8e76-6acf-4616-927d-fb5ff40dde6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:02.491173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c8e76-6acf-4616-927d-fb5ff40dde6c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:02.498302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 469/953] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:18:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:02.506209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:02.617581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c9d0a-5284-4d38-b908-18be05bb7b80 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:02.648038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4ba7-b57c-4856-bb28-a479503d1e55 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.648385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4ba7-b57c-4856-bb28-a479503d1e55 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.648651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4ba7-b57c-4856-bb28-a479503d1e55 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.650987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 470/954] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:18:02.655645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:02.656321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.656646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.657153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.657621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:02.657929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:02.658921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621045d-12e3-4daa-8bec-ae64b68597a1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.659294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:02.659667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621045d-12e3-4daa-8bec-ae64b68597a1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.660137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621045d-12e3-4daa-8bec-ae64b68597a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.665314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 485/955] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:18:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:02.674068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:02.679545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3621045d-12e3-4daa-8bec-ae64b68597a1 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:02.680934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3621045d-12e3-4daa-8bec-ae64b68597a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:02.685455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:02.727996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:02.752441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 471/956] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:02.760828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564509d2-18ce-47f1-834b-0418d6ca3ff8 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.760828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564509d2-18ce-47f1-834b-0418d6ca3ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.760828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564509d2-18ce-47f1-834b-0418d6ca3ff8 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.765239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 472/957] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:02.767594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edd4b6-c912-4962-a814-10b101b3e8ab None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.767885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edd4b6-c912-4962-a814-10b101b3e8ab None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.768158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edd4b6-c912-4962-a814-10b101b3e8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.785744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-10edd4b6-c912-4962-a814-10b101b3e8ab None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:02.787767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-10edd4b6-c912-4962-a814-10b101b3e8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:02.792263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcbd2d-426e-43a1-8f8b-a0bd0c98a718 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:02.829449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d19780-04d0-46ac-9add-a0d0fc9ef6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:02.830108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d19780-04d0-46ac-9add-a0d0fc9ef6af None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:02.830394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d19780-04d0-46ac-9add-a0d0fc9ef6af None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:02.830673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d19780-04d0-46ac-9add-a0d0fc9ef6af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:02.831413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d19780-04d0-46ac-9add-a0d0fc9ef6af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:02.832787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d19780-04d0-46ac-9add-a0d0fc9ef6af None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:02.862378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 473/958] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:02.863214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edd4b6-c912-4962-a814-10b101b3e8ab None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:02.905215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 486/959] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:18:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:02.905215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d19780-04d0-46ac-9add-a0d0fc9ef6af None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:02.983964 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:02.988514 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edd4b6-c912-4962-a814-10b101b3e8ab None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:03.002271 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:03.038907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc2c71-b79a-4fa1-b828-414dbd9803ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:03.039837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2c71-b79a-4fa1-b828-414dbd9803ab None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:03.040194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2c71-b79a-4fa1-b828-414dbd9803ab None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:03.040544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2c71-b79a-4fa1-b828-414dbd9803ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:03.040722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d19780-04d0-46ac-9add-a0d0fc9ef6af None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:03.053028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc2c71-b79a-4fa1-b828-414dbd9803ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-efcc2c71-b79a-4fa1-b828-414dbd9803ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:03.063628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:03.064867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 474/960] 10.222.0.41 () {56 vars in 1120 bytes} [Sat May 28 03:18:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:03.073138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc2c71-b79a-4fa1-b828-414dbd9803ab None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:03.089928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:03.090757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:03.091045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:03.091392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:03.092117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:03.093574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:03.110812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:03.111899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:03.112727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:03.113476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 487/961] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:18:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:03.119182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:03.227343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc2c71-b79a-4fa1-b828-414dbd9803ab None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:03.238078 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073c6439-6e27-4ebe-8a28-80df360195a7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:03.266065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:03.274777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:03.275803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:03.276036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:03.276475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:03.276745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:03.276850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:03.277027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:03.277563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:03.277864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:03.278148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:03.278949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:03.279455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:03.279559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:03.281164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:03.286586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 488/962] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:18:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:03.291547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 475/963] 10.222.0.41 () {56 vars in 1115 bytes} [Sat May 28 03:18:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:03.294021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:03.295250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:03.324919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:03.402919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea9da5-b47c-4065-b7a7-63f228808bbb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:03.420077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830cf4c-9723-4d9b-8c76-ad4441140346 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:03.438829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:03.439622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:03.440008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:03.440385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:03.441238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:03.441649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:03.443472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:03.456844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 489/964] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:18:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:03.461059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:03.462146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:03.462579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:03.463001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:03.476174 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:03.478002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:03.498213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:03.499340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:03.500447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:03.500447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:03.500447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:03.500447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:03.502365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 476/965] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:03.508185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:03.571694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:03.625842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954859cc-45d3-4ff2-844d-0414909d7620 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:03.662440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:03.663102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:03.663390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:03.663669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:03.664391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:03.664700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:03.666097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:03.680198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 490/966] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:03.685655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:03.774022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:03.797963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a928-14fc-4819-b90f-0a6f3311ae69 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:03.837622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:03.838444 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:03.838731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:03.839011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:03.847549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:03.849076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:18:03.864442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:03.865484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:03.866405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 477/967] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:18:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:03.874205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:03.991513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6351dd-0edc-4fa3-a998-20f39b76e09b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:04.018430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:04.032015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69c825-e35b-4d00-8f0b-401cfe122c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:04.032674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c825-e35b-4d00-8f0b-401cfe122c87 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.032958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c825-e35b-4d00-8f0b-401cfe122c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.033272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69c825-e35b-4d00-8f0b-401cfe122c87 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.040649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69c825-e35b-4d00-8f0b-401cfe122c87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:04.040649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69c825-e35b-4d00-8f0b-401cfe122c87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:04.040649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69c825-e35b-4d00-8f0b-401cfe122c87 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:04.055522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 491/968] 10.222.0.41 () {56 vars in 1149 bytes} [Sat May 28 03:18:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:04.061590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:04.174955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43d98e-49f7-4306-aa8e-73bde00c0362 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:04.211991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba2041-081f-480b-8dc5-d68f59b9f7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:04.212645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba2041-081f-480b-8dc5-d68f59b9f7be None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.212934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba2041-081f-480b-8dc5-d68f59b9f7be None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.213144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b171c-b40c-4511-9386-a5363c391b6f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.213329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba2041-081f-480b-8dc5-d68f59b9f7be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.213501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b171c-b40c-4511-9386-a5363c391b6f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.213781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b171c-b40c-4511-9386-a5363c391b6f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.215932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 492/969] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:04.220977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899148f2-6b7a-4c7f-b862-77e710c11d1d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.221304 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899148f2-6b7a-4c7f-b862-77e710c11d1d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.221589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899148f2-6b7a-4c7f-b862-77e710c11d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.225525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:04.241853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-899148f2-6b7a-4c7f-b862-77e710c11d1d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:04.246438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-899148f2-6b7a-4c7f-b862-77e710c11d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:04.285801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba2041-081f-480b-8dc5-d68f59b9f7be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:04.287215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba2041-081f-480b-8dc5-d68f59b9f7be None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:04.325005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 493/970] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:04.350491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899148f2-6b7a-4c7f-b862-77e710c11d1d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:04.403985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 478/971] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:04 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:18:04.409092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba2041-081f-480b-8dc5-d68f59b9f7be None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:04.450304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:04.475269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:04.479023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899148f2-6b7a-4c7f-b862-77e710c11d1d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:04.511434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba2041-081f-480b-8dc5-d68f59b9f7be None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:04.515239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:04.516147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.516652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.517161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.518060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:04.519808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:04.521453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING keystone.server.flask.common [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 28 03:18:04.550007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:04.550007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.550007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.550007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.559170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:04.560870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:04.562256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.sql.core [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 03:18:04.562256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 28 03:18:04.562256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [parameters: {'user_id': '5cb3c7da31a047c8959184c65d6386ca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 28 03:18:04.562256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() May 28 03:18:04.572394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 03:18:04.575104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application yield resource May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.rollback() May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.force_reraise() May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise self.value May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application session.commit() May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 03:18:04.576059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._prepare_impl() May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.flush() May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._flush(objects) May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application compat.raise_( May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application flush_context.execute() May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rec.execute(self) May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 03:18:04.576953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = connection._execute_20( May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 03:18:04.577981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 03:18:04.579492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [parameters: {'user_id': '5cb3c7da31a047c8959184c65d6386ca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:04.581251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 28 03:18:04.582945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 28 03:18:04.584796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 28 03:18:04.584796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 28 03:18:04.584796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 03:18:04.584796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 03:18:04.584796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 28 03:18:04.584796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:04.585747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:04.587458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:04.589122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:04.589122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:04.589122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:04.589122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:04.589545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 494/972] 10.222.0.41 () {60 vars in 1131 bytes} [Sat May 28 03:18:04 2022] POST /identity/v3/users => generated 160 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:04.589671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 479/973] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:04.595111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:04.595714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:04.698370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:04.710966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc5624-9f9d-436c-85be-7f8989ad28bb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:04.729778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78d8fe-6b39-456a-9df9-1567d9393b1d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:04.756599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:04.757430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.757720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.758002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.767518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:04.768982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:04.769912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:04.770588 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.771154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.771438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.772784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:04.773097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:04.774705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:18:04.786361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:04.787317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 03:18:04.788250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:04.790850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 495/974] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:04.795599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 480/975] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:18:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:04.796349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:04.796768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:04.826933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:04.929126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81218b-dd37-41a4-ad55-5168fb5dc31c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:04.935178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed90a87-c445-4e23-89ee-78355693dd7a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:04.971981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:04.972924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.973260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.973541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.974345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:04.974655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:04.975685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:04.976087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:04.976911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:04.977436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:04.977919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:04.985923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:04.987612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:18:05.002589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:05.003516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:05.004424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:18:05.004424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:18:05.004424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:18:05.004424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:18:05.004424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:18:05.004424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:18:05.004424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:05.004424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:05.007057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 481/976] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:18:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:05.014640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:05.024436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 496/977] 10.222.0.41 () {56 vars in 1101 bytes} [Sat May 28 03:18:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:05.028835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:05.137882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fdf09-7637-4012-bd4a-b2c6cb9e03c7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:05.179239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ea7b5c-dd00-4f90-9d10-7b111cf27329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:05.179911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7b5c-dd00-4f90-9d10-7b111cf27329 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.180204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7b5c-dd00-4f90-9d10-7b111cf27329 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.180487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ea7b5c-dd00-4f90-9d10-7b111cf27329 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.181327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea7b5c-dd00-4f90-9d10-7b111cf27329 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:05.181653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea7b5c-dd00-4f90-9d10-7b111cf27329 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:05.183154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ea7b5c-dd00-4f90-9d10-7b111cf27329 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:05.198172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 482/978] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:18:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:05.204713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:05.292739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:05.313915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5475d9a-12a8-43d3-9e0f-51d45e241b1a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:05.330194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264acf04-94d9-48a3-9900-ee1a5f612ff7 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.330530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264acf04-94d9-48a3-9900-ee1a5f612ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.330813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264acf04-94d9-48a3-9900-ee1a5f612ff7 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.332493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 483/979] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:18:05.338609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84991be7-2e1a-4223-bf1f-cb35df1accf0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.338995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84991be7-2e1a-4223-bf1f-cb35df1accf0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.339290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84991be7-2e1a-4223-bf1f-cb35df1accf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.356332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:05.356996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.357310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.357588 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.358419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] RBAC: Adding query filter params (role.id=83d48564c08d484481c9ec8706230ee6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=e619c4b1fa304a8a99de2f60d1b6cc89) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:05.358731 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:05.359177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-84991be7-2e1a-4223-bf1f-cb35df1accf0 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:05.360166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:05.360541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-84991be7-2e1a-4223-bf1f-cb35df1accf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:05.367616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 497/980] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:05 2022] GET /identity/v3/role_assignments?user.id=e619c4b1fa304a8a99de2f60d1b6cc89&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=83d48564c08d484481c9ec8706230ee6 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:05.434900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:05.440630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 484/981] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:05.463169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:05.573853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fed759-c364-4aa7-b7e6-2ec1fc937d20 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:05.613939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edab68f-cf32-46eb-96c9-fe1ff3df3a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:05.614762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab68f-cf32-46eb-96c9-fe1ff3df3a3e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.615043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab68f-cf32-46eb-96c9-fe1ff3df3a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.615312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab68f-cf32-46eb-96c9-fe1ff3df3a3e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.615966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edab68f-cf32-46eb-96c9-fe1ff3df3a3e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:05.617383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edab68f-cf32-46eb-96c9-fe1ff3df3a3e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7edab68f-cf32-46eb-96c9-fe1ff3df3a3e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:05.630782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:05.631801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:05.632640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 498/982] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:18:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:05.638509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84991be7-2e1a-4223-bf1f-cb35df1accf0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:05.738548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84991be7-2e1a-4223-bf1f-cb35df1accf0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:05.775761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:05.778311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.778311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.778311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.778311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:05.778311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:05.779429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:05.782184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a85d4-8e96-4efd-aa32-1d0141828e81 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.782471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a85d4-8e96-4efd-aa32-1d0141828e81 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.782757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a85d4-8e96-4efd-aa32-1d0141828e81 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.784808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 499/984] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:05.789088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 485/984] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:18:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:05.790133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:05.790277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccdeb5-6383-461c-a820-5c6b5aee6400 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.790682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccdeb5-6383-461c-a820-5c6b5aee6400 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.790968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccdeb5-6383-461c-a820-5c6b5aee6400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.809334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-50ccdeb5-6383-461c-a820-5c6b5aee6400 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:05.810821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-50ccdeb5-6383-461c-a820-5c6b5aee6400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:05.812796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:05.853170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:05.879360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 500/985] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:05.891965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ea078-dd55-4a50-9923-f89ce249022a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:05.904503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccdeb5-6383-461c-a820-5c6b5aee6400 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:05.934710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:05.935461 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:05.935751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:05.936029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:05.936736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:05.937048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:05.938451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:05.943808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 486/986] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:18:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:05.949906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:06.010215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccdeb5-6383-461c-a820-5c6b5aee6400 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:06.045078 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535a576-b27c-4cd6-b08b-871fe29677e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:06.045990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a576-b27c-4cd6-b08b-871fe29677e3 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:06.046276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a576-b27c-4cd6-b08b-871fe29677e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:06.046560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a576-b27c-4cd6-b08b-871fe29677e3 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:06.047341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535a576-b27c-4cd6-b08b-871fe29677e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:06.048719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535a576-b27c-4cd6-b08b-871fe29677e3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:06.053689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8535ca1-14ac-4af7-aebf-bbaa0c4a8880 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:06.089697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:06.097654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:06.098314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:06.098600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:06.098880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:06.099678 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] RBAC: Adding query filter params (interface=admin, service_id=b7e3ae5d60c24c9bbfb358438853b95b, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:06.099993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:06.101394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:06.105278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:06.111525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 487/987] 10.222.0.41 () {56 vars in 1249 bytes} [Sat May 28 03:18:05 2022] GET /identity/v3/endpoints?service_id=b7e3ae5d60c24c9bbfb358438853b95b&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:06.112365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:06.112875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 501/988] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:18:05 2022] POST /identity/v3/services => generated 242 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:06.236835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522c1f-46ca-4970-8786-399e7354bb4a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:06.279674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93eccb-228c-4a63-bdd5-2f701428d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:06.280267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93eccb-228c-4a63-bdd5-2f701428d882 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:06.280553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93eccb-228c-4a63-bdd5-2f701428d882 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:06.280829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93eccb-228c-4a63-bdd5-2f701428d882 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:06.281549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93eccb-228c-4a63-bdd5-2f701428d882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:06.282943 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93eccb-228c-4a63-bdd5-2f701428d882 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:06.289861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 488/989] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:18:06 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:18:07.631356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c3b5-05d5-4992-b503-5118f2a292ba None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:07.631905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c3b5-05d5-4992-b503-5118f2a292ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:07.631905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771c3b5-05d5-4992-b503-5118f2a292ba None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:07.633648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 502/990] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:07.638707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72449f99-ce0d-49b0-8300-420bfaae13ea None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:07.639142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72449f99-ce0d-49b0-8300-420bfaae13ea None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:07.639322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72449f99-ce0d-49b0-8300-420bfaae13ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:07.655877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-72449f99-ce0d-49b0-8300-420bfaae13ea None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:07.657053 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-72449f99-ce0d-49b0-8300-420bfaae13ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:07.703972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:07.724236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 489/991] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:07.747661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771c3b5-05d5-4992-b503-5118f2a292ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:07.849721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771c3b5-05d5-4992-b503-5118f2a292ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:07.886248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:07.886995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:07.887279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:07.887556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] PATH_INFO: `/v3/services/image` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:07.888207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:07.889596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:07.905001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:07.906185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:07.907041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:07.908257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 503/992] 10.222.0.41 () {56 vars in 1105 bytes} [Sat May 28 03:18:07 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:07.911701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72449f99-ce0d-49b0-8300-420bfaae13ea None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:08.015552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72449f99-ce0d-49b0-8300-420bfaae13ea None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:08.049715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7190430-b327-4b79-a432-ae939bd54404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:08.050378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7190430-b327-4b79-a432-ae939bd54404 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.050721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7190430-b327-4b79-a432-ae939bd54404 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.050921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7190430-b327-4b79-a432-ae939bd54404 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.051619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7190430-b327-4b79-a432-ae939bd54404 None admin] RBAC: Adding query filter params (name=image) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:08.051921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7190430-b327-4b79-a432-ae939bd54404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:08.053329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7190430-b327-4b79-a432-ae939bd54404 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:08.062756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 490/993] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:08.063865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:08.125775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:08.165345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77802480-5e4f-4811-b6c3-48c804cddd64 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:08.198357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8017ef-baa1-4a14-aa6a-26ebaae8ff76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:08.199067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8017ef-baa1-4a14-aa6a-26ebaae8ff76 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.199358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8017ef-baa1-4a14-aa6a-26ebaae8ff76 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.199634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8017ef-baa1-4a14-aa6a-26ebaae8ff76 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.200362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8017ef-baa1-4a14-aa6a-26ebaae8ff76 None admin] RBAC: Adding query filter params (type=image) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:08.200673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8017ef-baa1-4a14-aa6a-26ebaae8ff76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:08.202117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8017ef-baa1-4a14-aa6a-26ebaae8ff76 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:08.213489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 504/994] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:08.404957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fadd2-6097-4d01-860e-2dac5cd1db69 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.405283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fadd2-6097-4d01-860e-2dac5cd1db69 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.405571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fadd2-6097-4d01-860e-2dac5cd1db69 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.406941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 491/995] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:08.412448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75bb58-6eaa-4054-b662-f18f1b3347b6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.412738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75bb58-6eaa-4054-b662-f18f1b3347b6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.413018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75bb58-6eaa-4054-b662-f18f1b3347b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.430113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ff75bb58-6eaa-4054-b662-f18f1b3347b6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:08.431218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ff75bb58-6eaa-4054-b662-f18f1b3347b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:08.494429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed3291-0e5e-4585-9c0d-957acba5c8f7 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.494775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed3291-0e5e-4585-9c0d-957acba5c8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.494986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed3291-0e5e-4585-9c0d-957acba5c8f7 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.496608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 492/996] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:08.500716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 505/997] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:08.501701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a719b1-5a06-41b8-abf6-1b45c08b1b7c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.501992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a719b1-5a06-41b8-abf6-1b45c08b1b7c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.502270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a719b1-5a06-41b8-abf6-1b45c08b1b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.519261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b5a719b1-5a06-41b8-abf6-1b45c08b1b7c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:08.520427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b5a719b1-5a06-41b8-abf6-1b45c08b1b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:08.524430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75bb58-6eaa-4054-b662-f18f1b3347b6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:08.564908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:08.588937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:08.589547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 493/998] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:08.612602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a719b1-5a06-41b8-abf6-1b45c08b1b7c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:08.637560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75bb58-6eaa-4054-b662-f18f1b3347b6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:08.673564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:08.674359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.674870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.675000 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.675545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:08.676917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:08.689425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:08.690404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:08.691256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:08.695708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 506/999] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:18:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:08.696761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:08.720041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a719b1-5a06-41b8-abf6-1b45c08b1b7c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:08.754247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:08.755041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.755325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.755597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.756244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:08.757647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:08.770172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:08.771195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:08.772030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 494/1000] 10.222.0.41 () {56 vars in 1126 bytes} [Sat May 28 03:18:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:08.777267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:08.805326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb5639-d588-4300-942b-bbbe44285117 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:08.822252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:08.827450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:08.841641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:08.842298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.842580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.842863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.843543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:08.843846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:08.845220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:08.850260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 507/1001] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:18:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:08.855728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:08.879472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849613c5-e836-4319-a461-e1884ccbddfe None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:08.920663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:08.921358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:08.921642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:08.921917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:08.922612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:08.922918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:08.924283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:08.929191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 495/1002] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:18:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:08.936732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:08.969629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f9d60-2ca7-4dae-b7ee-0b05ed788076 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:09.008042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:09.008698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:09.008978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:09.009283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:09.009985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:09.010291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:09.012452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:09.018620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 508/1003] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:18:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:09.024254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:09.048998 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49b2a9-5f2c-4afc-b764-110761b2f0c6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:09.051360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:09.075958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:09.082881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:09.083546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:09.083892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:09.084103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:09.084804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:09.085131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:09.086642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:09.096445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 496/1004] 10.222.0.41 () {56 vars in 1122 bytes} [Sat May 28 03:18:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:09.097621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:09.135636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a1678-089d-4adb-b77a-51be10a7568f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:09.171382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:09.172138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:09.172421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:09.172695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:09.173611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] RBAC: Adding query filter params (interface=public, service_id=9417e438e94742948ba7ef41e86a1b0b, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:09.173925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:09.175827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:09.190204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 509/1005] 10.222.0.41 () {56 vars in 1251 bytes} [Sat May 28 03:18:09 2022] GET /identity/v3/endpoints?service_id=9417e438e94742948ba7ef41e86a1b0b&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:09.191170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:09.199078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec03440-c275-40f0-82db-31b5c928b0e7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:09.236664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083b97e-220a-4ed5-8526-25689c9ac72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:09.237276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b97e-220a-4ed5-8526-25689c9ac72f None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:09.237556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b97e-220a-4ed5-8526-25689c9ac72f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:09.237825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b97e-220a-4ed5-8526-25689c9ac72f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:09.238576 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083b97e-220a-4ed5-8526-25689c9ac72f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:09.239919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083b97e-220a-4ed5-8526-25689c9ac72f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:09.291330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc356ff3-f9c5-423a-a119-f25ed30abcb6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:09.304789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:09.313151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 497/1006] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:18:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:18:09.334679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc879047-6ec7-4900-a041-8ddfb659adb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:09.335377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc879047-6ec7-4900-a041-8ddfb659adb6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:09.335663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc879047-6ec7-4900-a041-8ddfb659adb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:09.335942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc879047-6ec7-4900-a041-8ddfb659adb6 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:09.336656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc879047-6ec7-4900-a041-8ddfb659adb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:09.338166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc879047-6ec7-4900-a041-8ddfb659adb6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:09.345892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 510/1007] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:18:09 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:18:10.474111 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f6e32-dff8-44bf-840a-a75f215076bd None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:10.474661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f6e32-dff8-44bf-840a-a75f215076bd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:10.474661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f6e32-dff8-44bf-840a-a75f215076bd None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:10.476244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 498/1008] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:10.481681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7c0b-5533-4a82-84b6-45b049cc6c93 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:10.482112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7c0b-5533-4a82-84b6-45b049cc6c93 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:10.482289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd7c0b-5533-4a82-84b6-45b049cc6c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:10.501154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0bbd7c0b-5533-4a82-84b6-45b049cc6c93 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:10.501320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0bbd7c0b-5533-4a82-84b6-45b049cc6c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:10.568978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 511/1009] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:10.591057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f6e32-dff8-44bf-840a-a75f215076bd None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:10.620331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:10.694089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f6e32-dff8-44bf-840a-a75f215076bd None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:10.728564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f8c35-750c-4f25-88f8-f0cdef5a3409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:10.729179 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f8c35-750c-4f25-88f8-f0cdef5a3409 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:10.729501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f8c35-750c-4f25-88f8-f0cdef5a3409 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:10.729737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f8c35-750c-4f25-88f8-f0cdef5a3409 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:10.730522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f8c35-750c-4f25-88f8-f0cdef5a3409 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:10.731895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f8c35-750c-4f25-88f8-f0cdef5a3409 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:10.825137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 499/1010] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:18:10 2022] POST /identity/v3/services => generated 242 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:11.573660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9cf51-f371-49a0-ad48-94ef14327c72 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:11.574816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3be4e9-c267-414e-9fef-32cf051a4be3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:11.574816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3be4e9-c267-414e-9fef-32cf051a4be3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:11.575210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9cf51-f371-49a0-ad48-94ef14327c72 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:11.575210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9cf51-f371-49a0-ad48-94ef14327c72 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:11.575376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3be4e9-c267-414e-9fef-32cf051a4be3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:11.576212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 500/1011] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:11.576370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 512/1012] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:11.581379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63c73-4ad9-4502-a5f7-23b76d1d76e8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:11.581670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63c73-4ad9-4502-a5f7-23b76d1d76e8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:11.581948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f63c73-4ad9-4502-a5f7-23b76d1d76e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:11.582045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b254a-99de-4ba9-8939-d95a074d87d4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:11.582237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b254a-99de-4ba9-8939-d95a074d87d4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:11.582517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b254a-99de-4ba9-8939-d95a074d87d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:11.599583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-618b254a-99de-4ba9-8939-d95a074d87d4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:11.599886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-23f63c73-4ad9-4502-a5f7-23b76d1d76e8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:11.600763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-618b254a-99de-4ba9-8939-d95a074d87d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:11.601024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-23f63c73-4ad9-4502-a5f7-23b76d1d76e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:11.685388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 501/1013] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:11 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:11.691799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 513/1014] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:11 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:11.704810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f63c73-4ad9-4502-a5f7-23b76d1d76e8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:11.718232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b254a-99de-4ba9-8939-d95a074d87d4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:11.723553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:11.747287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:11.808030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f63c73-4ad9-4502-a5f7-23b76d1d76e8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:11.819823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b254a-99de-4ba9-8939-d95a074d87d4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:11.845622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:11.846348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:11.846638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:11.846907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:11.847822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:11.849234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:11.855155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ef13f-f046-4b87-ab70-a62082221310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:11.855986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ef13f-f046-4b87-ab70-a62082221310 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:11.856277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ef13f-f046-4b87-ab70-a62082221310 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:11.856536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6ef13f-f046-4b87-ab70-a62082221310 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:11.857722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ef13f-f046-4b87-ab70-a62082221310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:11.859137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6ef13f-f046-4b87-ab70-a62082221310 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:11.860954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 514/1015] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:11.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:11.867501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:11.868552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 502/1016] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:18:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:11.869371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ef13f-f046-4b87-ab70-a62082221310 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:11.871567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:11.961316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:11.978012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:11.983966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879bbf1-00dd-4727-a550-54d64121d3d5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:11.984210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6ef13f-f046-4b87-ab70-a62082221310 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:12.021019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:12.023180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:12.023180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:12.023180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:12.023180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:12.023517 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:12.024323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e0b96-2920-4979-8b7c-9bb886170e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:12.024645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:12.025914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0b96-2920-4979-8b7c-9bb886170e36 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:12.026906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0b96-2920-4979-8b7c-9bb886170e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:12.027478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0b96-2920-4979-8b7c-9bb886170e36 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:12.028688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0b96-2920-4979-8b7c-9bb886170e36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:12.029819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 503/1017] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:18:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:12.030537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0b96-2920-4979-8b7c-9bb886170e36 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:12.035297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:12.135612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba766e9-71e5-4f61-9cdd-54d2e5378720 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:12.168788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:12.169476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:12.169754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:12.170032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:12.170725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:12.171030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:12.172424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:12.177391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 504/1018] 10.222.0.41 () {56 vars in 1112 bytes} [Sat May 28 03:18:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:12.177639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 515/1019] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:18:11 2022] POST /identity/v3/projects => generated 311 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:12.191448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:12.202361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:12.299318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cc9d3-2cf3-430d-98e4-11b9b59bd063 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:12.335276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878e2c5-165a-454a-933c-7c683d545f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:12.335948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878e2c5-165a-454a-933c-7c683d545f02 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:12.336230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878e2c5-165a-454a-933c-7c683d545f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:12.336505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878e2c5-165a-454a-933c-7c683d545f02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:12.337305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878e2c5-165a-454a-933c-7c683d545f02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:12.338763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d878e2c5-165a-454a-933c-7c683d545f02 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:12.420484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 505/1020] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:18:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:18:13.045771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf6386-5faa-4238-9053-eb718874f681 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:13.046363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf6386-5faa-4238-9053-eb718874f681 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:13.046363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf6386-5faa-4238-9053-eb718874f681 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:13.047733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 516/1021] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:13.052863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33916c-d67a-47aa-8512-aca3f51efa2a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:13.053316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33916c-d67a-47aa-8512-aca3f51efa2a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:13.053544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33916c-d67a-47aa-8512-aca3f51efa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:13.071123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9b33916c-d67a-47aa-8512-aca3f51efa2a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:13.072293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9b33916c-d67a-47aa-8512-aca3f51efa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:13.129722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:13.145597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 506/1022] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:13.168873 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf6386-5faa-4238-9053-eb718874f681 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:13.275231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf6386-5faa-4238-9053-eb718874f681 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:13.309026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:13.309828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:13.310109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:13.310378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] PATH_INFO: `/v3/services/image` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:13.311017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:13.312375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:13.324266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:13.325396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:13.326262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:13.327894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 517/1023] 10.222.0.41 () {56 vars in 1105 bytes} [Sat May 28 03:18:13 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:13.331881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33916c-d67a-47aa-8512-aca3f51efa2a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:13.431773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33916c-d67a-47aa-8512-aca3f51efa2a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:13.465475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:13.466128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:13.466406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:13.466779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:13.467477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:13.467779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:13.469177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:13.478963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 507/1024] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:13.480102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:13.495715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:13.581322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160c6f4-2c57-4328-8e96-554fb0fe6429 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:13.615814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:13.617002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:13.617547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:13.618082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:13.619039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:13.619561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:13.621198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:13.626941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 518/1025] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:13.632897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:13.690342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:13.736989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43485e97-188b-4df8-ac3e-3ec69630e4a4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:13.771466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259046f-ed60-4e20-aa6f-653e209c8363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:13.772174 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259046f-ed60-4e20-aa6f-653e209c8363 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:13.772462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259046f-ed60-4e20-aa6f-653e209c8363 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:13.772738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259046f-ed60-4e20-aa6f-653e209c8363 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:13.773558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259046f-ed60-4e20-aa6f-653e209c8363 None admin] RBAC: Adding query filter params (interface=public, service_id=f3a2934e19364a24a861239ad1e61325, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:13.773871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259046f-ed60-4e20-aa6f-653e209c8363 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:13.775239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259046f-ed60-4e20-aa6f-653e209c8363 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:13.781198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 508/1026] 10.222.0.41 () {56 vars in 1251 bytes} [Sat May 28 03:18:13 2022] GET /identity/v3/endpoints?service_id=f3a2934e19364a24a861239ad1e61325&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:13.786101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:13.867121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:13.890289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6efa2f-77f6-4939-8712-ecb21b7fd1c8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:13.931029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aed7be-ca2c-4909-a3d3-953ae37d0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:13.931687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aed7be-ca2c-4909-a3d3-953ae37d0de9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:13.931965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aed7be-ca2c-4909-a3d3-953ae37d0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:13.932237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aed7be-ca2c-4909-a3d3-953ae37d0de9 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:13.932959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aed7be-ca2c-4909-a3d3-953ae37d0de9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:13.934414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aed7be-ca2c-4909-a3d3-953ae37d0de9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:13.941396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 519/1027] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:18:13 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:18:14.442999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799a313-0cde-4423-b584-a22361aca3d2 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:14.443520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799a313-0cde-4423-b584-a22361aca3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:14.443630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799a313-0cde-4423-b584-a22361aca3d2 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:14.445044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 509/1028] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:14.450692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491705e-406a-4d17-9cee-b20a63031ef9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:14.451075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491705e-406a-4d17-9cee-b20a63031ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:14.451251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491705e-406a-4d17-9cee-b20a63031ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:14.468810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c491705e-406a-4d17-9cee-b20a63031ef9 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:14.470020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c491705e-406a-4d17-9cee-b20a63031ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:14.539029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 520/1029] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:14.563773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799a313-0cde-4423-b584-a22361aca3d2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:14.666701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799a313-0cde-4423-b584-a22361aca3d2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:14.683087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:14.706142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b0d07-e55c-4fa5-a00e-82ed88d27e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:14.706142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b0d07-e55c-4fa5-a00e-82ed88d27e48 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:14.706142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b0d07-e55c-4fa5-a00e-82ed88d27e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:14.706142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b0d07-e55c-4fa5-a00e-82ed88d27e48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:14.706538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b0d07-e55c-4fa5-a00e-82ed88d27e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:14.706538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b0d07-e55c-4fa5-a00e-82ed88d27e48 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:14.711820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 510/1030] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:14.713040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491705e-406a-4d17-9cee-b20a63031ef9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:14.745638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d0b9e-e51d-422b-9d73-bc1a8303362d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:14.746803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d0b9e-e51d-422b-9d73-bc1a8303362d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:14.746939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d0b9e-e51d-422b-9d73-bc1a8303362d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:14.748349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 511/1031] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:14.754432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3407bd9-33a0-4eb8-b805-7b6621a21743 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:14.754994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3407bd9-33a0-4eb8-b805-7b6621a21743 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:14.755918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3407bd9-33a0-4eb8-b805-7b6621a21743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:14.776713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e3407bd9-33a0-4eb8-b805-7b6621a21743 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:14.778879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e3407bd9-33a0-4eb8-b805-7b6621a21743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:14.785157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:14.821185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491705e-406a-4d17-9cee-b20a63031ef9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:14.859032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8786c61-fb0b-4fee-915d-655fe1f5e2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:14.859701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786c61-fb0b-4fee-915d-655fe1f5e2c7 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:14.860213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786c61-fb0b-4fee-915d-655fe1f5e2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:14.860324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 512/1032] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:14.860405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786c61-fb0b-4fee-915d-655fe1f5e2c7 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:14.860969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8786c61-fb0b-4fee-915d-655fe1f5e2c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:14.862439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8786c61-fb0b-4fee-915d-655fe1f5e2c7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:14.883856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3407bd9-33a0-4eb8-b805-7b6621a21743 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:14.977220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 521/1033] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:18:14 2022] POST /identity/v3/users => generated 285 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:14.988467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3407bd9-33a0-4eb8-b805-7b6621a21743 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:15.006783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:15.034190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fe397-c8eb-4da6-a657-b0505a224654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:15.035078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe397-c8eb-4da6-a657-b0505a224654 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:15.035366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe397-c8eb-4da6-a657-b0505a224654 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:15.035648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe397-c8eb-4da6-a657-b0505a224654 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:15.048684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fe397-c8eb-4da6-a657-b0505a224654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-6b0fe397-c8eb-4da6-a657-b0505a224654 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:15.057714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:15.059403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:18:15.060286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:15.064933 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 513/1034] 10.222.0.41 () {56 vars in 1120 bytes} [Sat May 28 03:18:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:15.066349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8786c61-fb0b-4fee-915d-655fe1f5e2c7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:15.175702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8786c61-fb0b-4fee-915d-655fe1f5e2c7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:15.202755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:15.211296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:15.211963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:15.217494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:15.217494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:15.217494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:15.217494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:15.217494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:15.224424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 522/1035] 10.222.0.41 () {56 vars in 1115 bytes} [Sat May 28 03:18:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:15.233654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fe397-c8eb-4da6-a657-b0505a224654 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:15.340595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fe397-c8eb-4da6-a657-b0505a224654 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:15.374704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:15.375712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:15.376163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:15.376588 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:15.387404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:15.389042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:15.406848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:15.407931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:15.409059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:15.409059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:15.409059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:15.409059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:15.415319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 514/1036] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:15.416653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:15.519027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d5aee-509e-4793-88a2-fd5478b66b93 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:15.554545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:15.555208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:15.555491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:15.555764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:15.556474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:15.556779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:15.558166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:15.574207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 523/1037] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:15.574999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:15.603739 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:15.675223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8cf41-a041-4354-a603-32e483d2ab05 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:15.708381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:15.709200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:15.709487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:15.709761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:15.717495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:15.718946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:18:15.735269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:15.736287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:15.737281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 515/1038] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:18:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:15.743645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:15.795926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:15.847625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a030c-7226-4a9a-a71c-c8c26d6a2bea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:15.881062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4117a60-2efd-455c-a6db-d18194517d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:15.881761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4117a60-2efd-455c-a6db-d18194517d46 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:15.882044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4117a60-2efd-455c-a6db-d18194517d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:15.882320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4117a60-2efd-455c-a6db-d18194517d46 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:15.883151 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4117a60-2efd-455c-a6db-d18194517d46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:15.883466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4117a60-2efd-455c-a6db-d18194517d46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:15.884888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4117a60-2efd-455c-a6db-d18194517d46 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:15.907871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 524/1039] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:18:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:15.909220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.004191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:16.019452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720c03-2a2c-43bf-a99b-f79b6476e9a4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.054569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950cb1e-399c-4c64-8d20-98aaa0619ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.055143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950cb1e-399c-4c64-8d20-98aaa0619ebb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.055428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950cb1e-399c-4c64-8d20-98aaa0619ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.055702 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0950cb1e-399c-4c64-8d20-98aaa0619ebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.124695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950cb1e-399c-4c64-8d20-98aaa0619ebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.126197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0950cb1e-399c-4c64-8d20-98aaa0619ebb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.162807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3b33a-0a43-4d9f-b920-cf05ce03af18 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.163133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3b33a-0a43-4d9f-b920-cf05ce03af18 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.163391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3b33a-0a43-4d9f-b920-cf05ce03af18 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.164898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 525/1040] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:18:16.170675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5609a90-006e-4a0f-bc96-c920e2ad4a43 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.170965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5609a90-006e-4a0f-bc96-c920e2ad4a43 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.171246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5609a90-006e-4a0f-bc96-c920e2ad4a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.187661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f5609a90-006e-4a0f-bc96-c920e2ad4a43 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:16.189042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f5609a90-006e-4a0f-bc96-c920e2ad4a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:16.218635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:16.240808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 516/1041] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:15 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:18:16.242383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950cb1e-399c-4c64-8d20-98aaa0619ebb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.264925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 526/1042] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:16.288133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5609a90-006e-4a0f-bc96-c920e2ad4a43 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.330276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:16.343844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0950cb1e-399c-4c64-8d20-98aaa0619ebb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.377882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.378665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.378942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.379211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.388972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.390446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.397835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5609a90-006e-4a0f-bc96-c920e2ad4a43 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:16.407922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:16.409031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:16.410309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:16.410309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:16.410309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:16.410309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:16.413734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 517/1043] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:16.414586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.437260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.438546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.438840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.439113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.439763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.441371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.453352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:16.454327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:16.455310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:16.456159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 527/1044] 10.222.0.41 () {56 vars in 1105 bytes} [Sat May 28 03:18:16 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:16.461936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.517379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d1e6b-4d5d-4908-95bd-b699e8bce5c4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.553065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.553772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.554067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.554334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.555046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:16.555353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.556766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.561389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:16.565661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f832f63-b663-420d-8c73-950274e05fa4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.568129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 518/1045] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:16.574413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.601646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.602229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.602512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.602788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.603482 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:16.603786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.605183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.614764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 528/1046] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:16.615777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.682485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:16.688091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7450e8-9268-4284-b854-1d5a746aabac None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.720535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cfc65-4639-4c44-83af-e63572cd7bc8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.724591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.725443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.725727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.726001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.734443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.736052 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:18:16.750777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:16.751710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:16.752640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 519/1047] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:16.757700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.760920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.761648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.762023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.762301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.763063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] RBAC: Adding query filter params (type=image) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:16.763372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.764789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.774358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 529/1048] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:16.775932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.807823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:16.867294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30e5a-1734-462c-99c8-d112d9eb9562 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.887315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbd74b-7033-4f4d-8fd9-1a856031126c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:16.904481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.904481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.904481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.904481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.904885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:16.904885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.905975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.920046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 520/1049] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:16.927184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:16.935943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30e67d-0ce7-48fb-a5e2-2ce5c178c1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:16.936650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30e67d-0ce7-48fb-a5e2-2ce5c178c1b7 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:16.936940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30e67d-0ce7-48fb-a5e2-2ce5c178c1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:16.937260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30e67d-0ce7-48fb-a5e2-2ce5c178c1b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:16.938002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30e67d-0ce7-48fb-a5e2-2ce5c178c1b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:16.939366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd30e67d-0ce7-48fb-a5e2-2ce5c178c1b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:16.947990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:17.027656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade5681-6171-49b6-94de-af577145a98a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:17.029249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 530/1050] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:18:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:17.082668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfe175-e65b-4fd7-a3e6-b53273a786ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:17.083353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfe175-e65b-4fd7-a3e6-b53273a786ea None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:17.083630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfe175-e65b-4fd7-a3e6-b53273a786ea None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:17.083902 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfe175-e65b-4fd7-a3e6-b53273a786ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:17.084761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfe175-e65b-4fd7-a3e6-b53273a786ea None admin] RBAC: Adding query filter params (role.id=83d48564c08d484481c9ec8706230ee6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f7ebbd8eb36c4919aedd3498bd8bc483) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:17.085086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfe175-e65b-4fd7-a3e6-b53273a786ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:17.086689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfe175-e65b-4fd7-a3e6-b53273a786ea None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:17.101136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 521/1051] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:16 2022] GET /identity/v3/role_assignments?user.id=f7ebbd8eb36c4919aedd3498bd8bc483&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=83d48564c08d484481c9ec8706230ee6 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:17.278982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737a36-086c-4d26-9caf-f6e77c5d8fb7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:17.279363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737a36-086c-4d26-9caf-f6e77c5d8fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:17.279552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737a36-086c-4d26-9caf-f6e77c5d8fb7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:17.280930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 531/1052] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:17.286541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af061a-80a7-4892-96ed-e3f2bc001846 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:17.286831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af061a-80a7-4892-96ed-e3f2bc001846 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:17.287113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af061a-80a7-4892-96ed-e3f2bc001846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:17.304841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-29af061a-80a7-4892-96ed-e3f2bc001846 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:17.306323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-29af061a-80a7-4892-96ed-e3f2bc001846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:17.376082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 522/1053] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:17.399624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70737a36-086c-4d26-9caf-f6e77c5d8fb7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:17.488228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:17.504691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70737a36-086c-4d26-9caf-f6e77c5d8fb7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:17.540362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d071c-b6cc-4a97-9529-df0bb6289713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:17.541258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d071c-b6cc-4a97-9529-df0bb6289713 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:17.541642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d071c-b6cc-4a97-9529-df0bb6289713 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:17.542014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d071c-b6cc-4a97-9529-df0bb6289713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:17.552664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d071c-b6cc-4a97-9529-df0bb6289713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b73d071c-b6cc-4a97-9529-df0bb6289713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:17.561334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:17.562903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:17.564032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:17.569176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 532/1054] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:18:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:17.571442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af061a-80a7-4892-96ed-e3f2bc001846 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:17.582822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:17.671655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af061a-80a7-4892-96ed-e3f2bc001846 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:17.705259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:17.705916 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:17.706195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:17.706466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:17.707256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:17.707563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:17.709040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:17.717696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 523/1055] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:18:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:17.723454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d071c-b6cc-4a97-9529-df0bb6289713 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:17.827383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d071c-b6cc-4a97-9529-df0bb6289713 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:17.861787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89f0ea-3572-4bc1-a713-35677daa79b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:17.864359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f0ea-3572-4bc1-a713-35677daa79b7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:17.864645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f0ea-3572-4bc1-a713-35677daa79b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:17.864922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f0ea-3572-4bc1-a713-35677daa79b7 None admin] PATH_INFO: `/v3/users/9b85e77ff82b4b159abd17b158da16c6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:17.881692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89f0ea-3572-4bc1-a713-35677daa79b7 None admin] RBAC: Authorizing `identity:get_user(user_id=9b85e77ff82b4b159abd17b158da16c6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:17.883146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89f0ea-3572-4bc1-a713-35677daa79b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:17.889542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 533/1056] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:17 2022] GET /identity/v3/users/9b85e77ff82b4b159abd17b158da16c6 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:17.890754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:17.962505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:17.993453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361dc942-780e-4333-8d23-83e76e89f6c0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:18.026021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14681f-d8c6-40b8-9798-d34dae0018f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:18.026728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14681f-d8c6-40b8-9798-d34dae0018f4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:18.027625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14681f-d8c6-40b8-9798-d34dae0018f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:18.027964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14681f-d8c6-40b8-9798-d34dae0018f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:18.094118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14681f-d8c6-40b8-9798-d34dae0018f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:18.095488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14681f-d8c6-40b8-9798-d34dae0018f4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:18.160499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:18.197961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 524/1057] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:18:18.199666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89f0ea-3572-4bc1-a713-35677daa79b7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:18.202859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:18.306667 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89f0ea-3572-4bc1-a713-35677daa79b7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:18.342556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a97df-7a90-4cef-b4d7-eb7de0455c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:18.343419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a97df-7a90-4cef-b4d7-eb7de0455c22 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:18.343706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a97df-7a90-4cef-b4d7-eb7de0455c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:18.343983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a97df-7a90-4cef-b4d7-eb7de0455c22 None admin] PATH_INFO: `/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:18.345240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a97df-7a90-4cef-b4d7-eb7de0455c22 None admin] RBAC: Authorizing `identity:get_project(project_id=a83cc4f995d44b269ce2d3d59076c4f1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:18.346623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a97df-7a90-4cef-b4d7-eb7de0455c22 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:18.348792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 534/1058] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:18 2022] GET /identity/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:18.354623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14681f-d8c6-40b8-9798-d34dae0018f4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:18.450796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14681f-d8c6-40b8-9798-d34dae0018f4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:18.488816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b756f-e024-40d8-8438-61ee104c63c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:18.489499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b756f-e024-40d8-8438-61ee104c63c1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:18.489783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b756f-e024-40d8-8438-61ee104c63c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:18.490058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b756f-e024-40d8-8438-61ee104c63c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:18.490877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b756f-e024-40d8-8438-61ee104c63c1 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=a83cc4f995d44b269ce2d3d59076c4f1, user.id=9b85e77ff82b4b159abd17b158da16c6) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:18.491187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b756f-e024-40d8-8438-61ee104c63c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:18.492700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b756f-e024-40d8-8438-61ee104c63c1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:18.505140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 525/1059] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:18 2022] GET /identity/v3/role_assignments?user.id=9b85e77ff82b4b159abd17b158da16c6&scope.project.id=a83cc4f995d44b269ce2d3d59076c4f1&role.id=bb0756f8af694633b9743a958eb46e7a => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:19.298431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffb60-bf3e-40ce-a1a2-7f6c6703648e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.298955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffb60-bf3e-40ce-a1a2-7f6c6703648e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.299033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffb60-bf3e-40ce-a1a2-7f6c6703648e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.300506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 535/1060] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:19.305567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90292d84-2610-4847-86ac-2e608e5e351a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.305956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90292d84-2610-4847-86ac-2e608e5e351a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.306115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90292d84-2610-4847-86ac-2e608e5e351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.312945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47ab8a-a47b-4520-9cb9-33efa76ae9dd None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.313140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47ab8a-a47b-4520-9cb9-33efa76ae9dd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.313371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47ab8a-a47b-4520-9cb9-33efa76ae9dd None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.314707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 536/1061] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:19.319488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db53b38-e4b4-47ec-97b5-ff2dbe86bc82 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.319781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db53b38-e4b4-47ec-97b5-ff2dbe86bc82 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.320060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db53b38-e4b4-47ec-97b5-ff2dbe86bc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.324633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-90292d84-2610-4847-86ac-2e608e5e351a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:19.326045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-90292d84-2610-4847-86ac-2e608e5e351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:19.336293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1db53b38-e4b4-47ec-97b5-ff2dbe86bc82 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:19.337471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1db53b38-e4b4-47ec-97b5-ff2dbe86bc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:19.354481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:19.395001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 526/1062] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:19.410574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:19.417150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 537/1063] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:19.417482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90292d84-2610-4847-86ac-2e608e5e351a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:19.438435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db53b38-e4b4-47ec-97b5-ff2dbe86bc82 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:19.526097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90292d84-2610-4847-86ac-2e608e5e351a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:19.545691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db53b38-e4b4-47ec-97b5-ff2dbe86bc82 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:19.565929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219a936-4f19-42c1-a012-a4eb6be8163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:19.566502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219a936-4f19-42c1-a012-a4eb6be8163e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.567045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219a936-4f19-42c1-a012-a4eb6be8163e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.567045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219a936-4f19-42c1-a012-a4eb6be8163e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.583004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70c743-0cd5-4609-a348-1dbe4464e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:19.583813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70c743-0cd5-4609-a348-1dbe4464e7ea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.584097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70c743-0cd5-4609-a348-1dbe4464e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.584369 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70c743-0cd5-4609-a348-1dbe4464e7ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.596534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70c743-0cd5-4609-a348-1dbe4464e7ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:19.600604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ac70c743-0cd5-4609-a348-1dbe4464e7ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:19.606945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:19.608493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:18:19.609347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:19.611087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 538/1064] 10.222.0.41 () {56 vars in 1120 bytes} [Sat May 28 03:18:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:19.617020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70c743-0cd5-4609-a348-1dbe4464e7ea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:19.644048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219a936-4f19-42c1-a012-a4eb6be8163e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:19.645555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219a936-4f19-42c1-a012-a4eb6be8163e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:19.681749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:19.724753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70c743-0cd5-4609-a348-1dbe4464e7ea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:19.756054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 527/1065] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 03:18:19.762016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219a936-4f19-42c1-a012-a4eb6be8163e None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:19.762561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:19.763237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.763521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.763805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.764541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:19.764851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:19.766250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:19.780083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 539/1066] 10.222.0.41 () {56 vars in 1115 bytes} [Sat May 28 03:18:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:19.780801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:19.825752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:19.865844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219a936-4f19-42c1-a012-a4eb6be8163e None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:19.885919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e005b9c-db59-40d6-a4a2-351da318fa80 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:19.900107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:19.900896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.901209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.901493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.913136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:19.913698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:19.916391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:19.924788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:19.925997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:19.926328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:19.926604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:19.937204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:19.943139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:19.944134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:19.944134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:19.944134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:19.944134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:19.944368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:19.944441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 528/1067] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:19.944508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:19.944583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:19.959218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:19.960119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:19.961004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:19.961004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:19.961004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:19.961004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:19.961004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 540/1068] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:19.966015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.051203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7898ce-b2b4-49e2-abb5-ef7cf4a8db3a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:20.073200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:20.075018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982940f2-20ab-42fb-a390-cf66e3f76745 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:20.093173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:20.093767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.094055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.094335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.095115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:20.095439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:20.096827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:20.121148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 529/1069] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:20.121359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:20.121359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.121359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.121359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.121660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:20.121660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:20.121660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:20.128219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 541/1070] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:20.134171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.243290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cd03b-9701-4ed7-a082-c5a222f69df5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:20.280896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:20.281736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.282022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.282296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.290156 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:20.291622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:18:20.307024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:20.309176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:20.310138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 530/1071] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:18:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:20.314867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.351948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:20.420511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987466e-4ae6-47fb-ac99-937bd3784d37 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:20.456014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:20.456682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.456968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.457279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.458109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:20.458421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:20.459842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:20.476747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 542/1072] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:18:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:20.482999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.517954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:20.584533 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d7857-52a6-4588-9f98-27cbc40c29f1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:20.617576 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebab34-2fdc-4906-9727-2b9c94cb9222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:20.618289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebab34-2fdc-4906-9727-2b9c94cb9222 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.618570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebab34-2fdc-4906-9727-2b9c94cb9222 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.618842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebab34-2fdc-4906-9727-2b9c94cb9222 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.629139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebab34-2fdc-4906-9727-2b9c94cb9222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:20.630595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebab34-2fdc-4906-9727-2b9c94cb9222 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-13ebab34-2fdc-4906-9727-2b9c94cb9222 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:20.647534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:20.648622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:20.649589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:20.649589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:20.649589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:20.649589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:20.649589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 531/1073] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:20.654906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.704380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0038d4-ea64-4109-b223-c728319f5685 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.704672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0038d4-ea64-4109-b223-c728319f5685 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.704949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0038d4-ea64-4109-b223-c728319f5685 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.706362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 532/1074] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:18:20.711232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff70b4-9720-481a-8a84-df5681a5d178 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.711520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff70b4-9720-481a-8a84-df5681a5d178 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.711799 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff70b4-9720-481a-8a84-df5681a5d178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.728856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8cff70b4-9720-481a-8a84-df5681a5d178 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:20.730652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8cff70b4-9720-481a-8a84-df5681a5d178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:20.745658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:20.754784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af8d76-15a0-492c-97e0-c3cf4b2f87c8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:20.787995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:20.788653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.788935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.789239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.789956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:20.790259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:20.791609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:20.797020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 533/1075] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:20.807585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 543/1076] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:20.808768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff70b4-9720-481a-8a84-df5681a5d178 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.819277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.826997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:20.913876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff70b4-9720-481a-8a84-df5681a5d178 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:20.923160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac4fa2-1f36-43e4-bf2e-0dbfc8225c33 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:20.948040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:20.952397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.952397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.952397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.955998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188e905-52a9-4378-ad9e-ca793df5d048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:20.956139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:20.956788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188e905-52a9-4378-ad9e-ca793df5d048 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:20.957075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188e905-52a9-4378-ad9e-ca793df5d048 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:20.957387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5188e905-52a9-4378-ad9e-ca793df5d048 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:20.957650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:20.968047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5188e905-52a9-4378-ad9e-ca793df5d048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:18:20.971983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:20.973459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:20.974383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:18:20.974383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:18:20.974383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:18:20.974383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:18:20.974383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:18:20.974383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:18:20.974383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:20.974383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5188e905-52a9-4378-ad9e-ca793df5d048 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:20.976096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:20.977068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:20.978631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:20.979416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 534/1077] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:20.979498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 544/1078] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:20.981226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.983784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188e905-52a9-4378-ad9e-ca793df5d048 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:20.998305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:21.058182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:21.079639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2b6cf-ebf7-490b-b5de-412a0458c22f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:21.081253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5188e905-52a9-4378-ad9e-ca793df5d048 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:21.114917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:21.115146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:21.115582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:21.115682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:21.115868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:21.115975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:21.116147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:21.116238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:21.116891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:21.116992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:21.117232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:21.117339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:21.118608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:21.118699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:21.126616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 545/1079] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:18:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:21.130264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 535/1080] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:21.132660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:21.135734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:21.209986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:21.235211 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b856a-21b5-4a1e-940e-2affaf51ec5f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:21.237598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da373fa-a885-4e29-beda-6184d5cb12f3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:21.268568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9758c-6610-46a3-b0f6-587e5145d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:21.269486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9758c-6610-46a3-b0f6-587e5145d818 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:21.269768 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9758c-6610-46a3-b0f6-587e5145d818 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:21.270044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9758c-6610-46a3-b0f6-587e5145d818 None admin] PATH_INFO: `/v3/users/9b85e77ff82b4b159abd17b158da16c6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:21.270947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748478a-b292-4eec-a235-dd1cec9a8ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:21.271241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9758c-6610-46a3-b0f6-587e5145d818 None admin] RBAC: Authorizing `identity:get_user(user_id=9b85e77ff82b4b159abd17b158da16c6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:21.271820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748478a-b292-4eec-a235-dd1cec9a8ec9 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:21.272097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748478a-b292-4eec-a235-dd1cec9a8ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:21.272365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748478a-b292-4eec-a235-dd1cec9a8ec9 None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f7ebbd8eb36c4919aedd3498bd8bc483/roles/83d48564c08d484481c9ec8706230ee6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:21.272673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9758c-6610-46a3-b0f6-587e5145d818 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:21.274342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 546/1081] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:21 2022] GET /identity/v3/users/9b85e77ff82b4b159abd17b158da16c6 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:21.277783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748478a-b292-4eec-a235-dd1cec9a8ec9 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=f7ebbd8eb36c4919aedd3498bd8bc483, role_id=83d48564c08d484481c9ec8706230ee6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:21.279215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748478a-b292-4eec-a235-dd1cec9a8ec9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:21.280628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9758c-6610-46a3-b0f6-587e5145d818 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:21.348118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:21.354036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 536/1082] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:18:21 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/f7ebbd8eb36c4919aedd3498bd8bc483/roles/83d48564c08d484481c9ec8706230ee6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:18:21.384468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9758c-6610-46a3-b0f6-587e5145d818 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:21.422866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba55e2-1401-4c60-bddb-2cdd6eb2c999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:21.423745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba55e2-1401-4c60-bddb-2cdd6eb2c999 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:21.424027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba55e2-1401-4c60-bddb-2cdd6eb2c999 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:21.424303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba55e2-1401-4c60-bddb-2cdd6eb2c999 None admin] PATH_INFO: `/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:21.425527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba55e2-1401-4c60-bddb-2cdd6eb2c999 None admin] RBAC: Authorizing `identity:get_project(project_id=a83cc4f995d44b269ce2d3d59076c4f1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:21.427022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba55e2-1401-4c60-bddb-2cdd6eb2c999 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:21.428763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 547/1083] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:21 2022] GET /identity/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:21.435644 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748478a-b292-4eec-a235-dd1cec9a8ec9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:21.546075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748478a-b292-4eec-a235-dd1cec9a8ec9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:21.579689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7c0f0-f027-4209-bc0f-10a6c3ea1b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:21.580581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c0f0-f027-4209-bc0f-10a6c3ea1b8b None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:21.580861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c0f0-f027-4209-bc0f-10a6c3ea1b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:21.581157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c0f0-f027-4209-bc0f-10a6c3ea1b8b None admin] PATH_INFO: `/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1/users/9b85e77ff82b4b159abd17b158da16c6/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:21.583085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7c0f0-f027-4209-bc0f-10a6c3ea1b8b None admin] RBAC: Authorizing `identity:create_grant(project_id=a83cc4f995d44b269ce2d3d59076c4f1, user_id=9b85e77ff82b4b159abd17b158da16c6, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:21.584514 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7c0f0-f027-4209-bc0f-10a6c3ea1b8b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:21.644118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 537/1084] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:18:21 2022] PUT /identity/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1/users/9b85e77ff82b4b159abd17b158da16c6/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:18:23.594748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddd432-ec81-430a-96cf-c3b0bb865dc0 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:23.595283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddd432-ec81-430a-96cf-c3b0bb865dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:23.595283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddd432-ec81-430a-96cf-c3b0bb865dc0 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:23.596649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 548/1085] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:23.602143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3c333-c1ae-4a3d-87ee-9ad2ba093d76 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:23.602564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3c333-c1ae-4a3d-87ee-9ad2ba093d76 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:23.602829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3c333-c1ae-4a3d-87ee-9ad2ba093d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:23.619201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-61a3c333-c1ae-4a3d-87ee-9ad2ba093d76 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:23.620358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-61a3c333-c1ae-4a3d-87ee-9ad2ba093d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:23.671524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:23.685039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 538/1086] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:23.709302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddd432-ec81-430a-96cf-c3b0bb865dc0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:23.724296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:23.813941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddd432-ec81-430a-96cf-c3b0bb865dc0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:23.846890 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de18825-49a1-45e8-a01b-116e4b4a3790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:23.847884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18825-49a1-45e8-a01b-116e4b4a3790 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:23.848171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18825-49a1-45e8-a01b-116e4b4a3790 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:23.848450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de18825-49a1-45e8-a01b-116e4b4a3790 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:23.859257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de18825-49a1-45e8-a01b-116e4b4a3790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4de18825-49a1-45e8-a01b-116e4b4a3790 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:23.867577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:23.868500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:23.869508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 549/1087] 10.222.0.41 () {56 vars in 1120 bytes} [Sat May 28 03:18:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:23.875271 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3c333-c1ae-4a3d-87ee-9ad2ba093d76 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:23.896996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d81de-cc86-4c0c-9f07-21bed6ed9138 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:23.897328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d81de-cc86-4c0c-9f07-21bed6ed9138 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:23.897608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d81de-cc86-4c0c-9f07-21bed6ed9138 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:23.898995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 550/1088] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:18:23.904092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83875ff3-7c27-44f9-ae11-51dfecc831fd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:23.904380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83875ff3-7c27-44f9-ae11-51dfecc831fd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:23.904670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83875ff3-7c27-44f9-ae11-51dfecc831fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:23.922491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-83875ff3-7c27-44f9-ae11-51dfecc831fd None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:23.923660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-83875ff3-7c27-44f9-ae11-51dfecc831fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:23.976738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3c333-c1ae-4a3d-87ee-9ad2ba093d76 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:23.987596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:23.992352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 551/1089] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:24.010617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.011847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.012133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.012408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.013174 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:24.013486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.014915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.016448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83875ff3-7c27-44f9-ae11-51dfecc831fd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.023173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 539/1090] 10.222.0.41 () {56 vars in 1115 bytes} [Sat May 28 03:18:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:24.028606 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.163393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:24.212711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f527b8-0c0a-423e-82cd-1aad575cb69f None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.212927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83875ff3-7c27-44f9-ae11-51dfecc831fd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.246379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.247157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.247435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.247543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1948-7883-4f6f-9811-71149fdb0354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.247707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.248300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1948-7883-4f6f-9811-71149fdb0354 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.248653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1948-7883-4f6f-9811-71149fdb0354 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.248934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de1948-7883-4f6f-9811-71149fdb0354 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.257994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.259170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de1948-7883-4f6f-9811-71149fdb0354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:24.259437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c5de1948-7883-4f6f-9811-71149fdb0354 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:24.267378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:24.271954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:24.272996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 552/1091] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:24.274573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1948-7883-4f6f-9811-71149fdb0354 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:24.277887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:24.278997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:24.280149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:24.280149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:24.280149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:24.280149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:24.280149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 540/1092] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:24.284657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.336534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:24.384316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de1948-7883-4f6f-9811-71149fdb0354 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.386053 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24a01f-0346-4239-b78e-464a4912b440 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.388155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:24.417744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.418395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.418616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.418891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.419633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:24.419938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.420044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.420632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.420918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.421223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.421330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.421935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:24.422243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.423597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.429272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 553/1093] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:24.435894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 541/1094] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:24.436054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.440573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.535782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c79b5e-63b7-4bd9-b533-1e54bb7f76c9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.537594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e67147-1f74-4e35-96f2-77c6a910dbcc None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.557498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:24.569447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2c28c-46d3-42ea-861f-77d17d087a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.570318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2c28c-46d3-42ea-861f-77d17d087a91 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.570429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.570624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2c28c-46d3-42ea-861f-77d17d087a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.570895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2c28c-46d3-42ea-861f-77d17d087a91 None admin] PATH_INFO: `/v3/users/9b85e77ff82b4b159abd17b158da16c6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.571160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.571446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.571726 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.572068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2c28c-46d3-42ea-861f-77d17d087a91 None admin] RBAC: Authorizing `identity:get_user(user_id=9b85e77ff82b4b159abd17b158da16c6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.573524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2c28c-46d3-42ea-861f-77d17d087a91 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.575739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 554/1095] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/users/9b85e77ff82b4b159abd17b158da16c6 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:24.581298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.581437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2c28c-46d3-42ea-861f-77d17d087a91 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.582716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:18:24.596859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:24.598206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:24.599218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 542/1096] 10.222.0.41 () {56 vars in 1093 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:24.604035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.628065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:24.680356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2c28c-46d3-42ea-861f-77d17d087a91 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.702831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53e137-5929-497c-aefc-6707999b7aa9 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.715102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5422a-2551-4384-846c-47b3d9284071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.715746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5422a-2551-4384-846c-47b3d9284071 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.716024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5422a-2551-4384-846c-47b3d9284071 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.716299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5422a-2551-4384-846c-47b3d9284071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.737509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.738248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.738538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.738816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.739639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:24.739953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.741400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.758782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 543/1097] 10.222.0.41 () {56 vars in 1133 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:24.764970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.774499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:24.784721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5422a-2551-4384-846c-47b3d9284071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.788526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e5422a-2551-4384-846c-47b3d9284071 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.846736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:24.866106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b2bc-e66b-461d-a12e-594ccc041df2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:24.884326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 555/1098] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 03:18:24.890269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5422a-2551-4384-846c-47b3d9284071 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:24.900006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f92d3f-ea1e-4a7c-808f-529682fedf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:24.900651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f92d3f-ea1e-4a7c-808f-529682fedf13 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:24.900932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f92d3f-ea1e-4a7c-808f-529682fedf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:24.901235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f92d3f-ea1e-4a7c-808f-529682fedf13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:24.970837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f92d3f-ea1e-4a7c-808f-529682fedf13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:24.972228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f92d3f-ea1e-4a7c-808f-529682fedf13 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:24.978527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:24.988867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5422a-2551-4384-846c-47b3d9284071 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:25.022431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e9fe0-bbbe-41e6-9399-2c93e49e028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:25.023300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e9fe0-bbbe-41e6-9399-2c93e49e028f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:25.023579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e9fe0-bbbe-41e6-9399-2c93e49e028f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:25.023853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e9fe0-bbbe-41e6-9399-2c93e49e028f None admin] PATH_INFO: `/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:25.025015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e9fe0-bbbe-41e6-9399-2c93e49e028f None admin] RBAC: Authorizing `identity:get_project(project_id=a83cc4f995d44b269ce2d3d59076c4f1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:25.026462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e9fe0-bbbe-41e6-9399-2c93e49e028f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:25.028134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 556/1099] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:25.034131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e9fe0-bbbe-41e6-9399-2c93e49e028f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:25.049372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:25.068910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 544/1100] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 03:18:25.074675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f92d3f-ea1e-4a7c-808f-529682fedf13 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:25.134755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e9fe0-bbbe-41e6-9399-2c93e49e028f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:25.170601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:25.171246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:25.171526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:25.171805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:25.172615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=a83cc4f995d44b269ce2d3d59076c4f1, user.id=9b85e77ff82b4b159abd17b158da16c6) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:25.172927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:25.174374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:25.179289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f92d3f-ea1e-4a7c-808f-529682fedf13 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:25.179994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 557/1101] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:25 2022] GET /identity/v3/role_assignments?user.id=9b85e77ff82b4b159abd17b158da16c6&scope.project.id=a83cc4f995d44b269ce2d3d59076c4f1&role.id=bb0756f8af694633b9743a958eb46e7a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:25.215074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:25.215893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:25.216177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:25.216454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:25.228096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:25.229518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:25.248759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:25.249831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:25.250760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:18:25.250760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:25.250760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:18:25.250760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:25.253264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 545/1102] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 03:18:25.255908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:25.282191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:25.364732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd2bdf-669d-4d89-ab3d-b5f3eb36fbd8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:25.399912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:25.400575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:25.400866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:25.401216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:25.402457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:25.402773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:25.404137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:25.415655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 558/1103] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:25.421803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:25.460392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:25.527803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0717a0-502a-459b-b13a-213cfe95e605 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:25.560613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:25.561431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:25.561716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:25.561989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:25.568507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:25.569984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:18:25.583926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:25.585007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:25.591526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 546/1104] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:18:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:25.592169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:25.669984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:25.687073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e9e70-4787-44cd-a8cc-8559e1cb61d6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:25.720572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e6ad2-630e-4471-abe7-e9d4002723c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:25.721265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6ad2-630e-4471-abe7-e9d4002723c4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:25.721549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6ad2-630e-4471-abe7-e9d4002723c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:25.721824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e6ad2-630e-4471-abe7-e9d4002723c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:25.722852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e6ad2-630e-4471-abe7-e9d4002723c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:25.723160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e6ad2-630e-4471-abe7-e9d4002723c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:25.724576 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e6ad2-630e-4471-abe7-e9d4002723c4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:25.736953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 559/1105] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:25.743170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:25.833172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:25.842098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fc875-a40b-49fa-b3ca-cc82b0468f30 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:25.878084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59729ff-9cd4-4106-9edb-98c616394136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:25.878734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59729ff-9cd4-4106-9edb-98c616394136 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:25.879025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59729ff-9cd4-4106-9edb-98c616394136 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:25.879307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59729ff-9cd4-4106-9edb-98c616394136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:25.880135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59729ff-9cd4-4106-9edb-98c616394136 None admin] RBAC: Adding query filter params (role.id=83d48564c08d484481c9ec8706230ee6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=f7ebbd8eb36c4919aedd3498bd8bc483) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:25.880451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59729ff-9cd4-4106-9edb-98c616394136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:25.881926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59729ff-9cd4-4106-9edb-98c616394136 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:25.888390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 547/1106] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:25 2022] GET /identity/v3/role_assignments?user.id=f7ebbd8eb36c4919aedd3498bd8bc483&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=83d48564c08d484481c9ec8706230ee6 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:27.402945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465ac0-f9c7-46bb-a136-5cb566791a8e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:27.403504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465ac0-f9c7-46bb-a136-5cb566791a8e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:27.403504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca465ac0-f9c7-46bb-a136-5cb566791a8e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:27.404857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 560/1107] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:27.410322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b493e6e-be4e-4ac7-801f-7d4e89a13b21 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:27.410738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b493e6e-be4e-4ac7-801f-7d4e89a13b21 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:27.410880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b493e6e-be4e-4ac7-801f-7d4e89a13b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:27.427948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4b493e6e-be4e-4ac7-801f-7d4e89a13b21 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:27.429173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4b493e6e-be4e-4ac7-801f-7d4e89a13b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:27.493701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 548/1108] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:27.517436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca465ac0-f9c7-46bb-a136-5cb566791a8e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:27.615469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca465ac0-f9c7-46bb-a136-5cb566791a8e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:27.647624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb5c48-9c08-44b1-b213-ad6490df6b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:27.648390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5c48-9c08-44b1-b213-ad6490df6b43 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:27.648674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5c48-9c08-44b1-b213-ad6490df6b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:27.648949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb5c48-9c08-44b1-b213-ad6490df6b43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:27.650130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb5c48-9c08-44b1-b213-ad6490df6b43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:27.651542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb5c48-9c08-44b1-b213-ad6490df6b43 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:27.653706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 561/1109] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:27.659091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b493e6e-be4e-4ac7-801f-7d4e89a13b21 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:27.740455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:27.754194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b493e6e-be4e-4ac7-801f-7d4e89a13b21 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:27.786494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50067c0d-b1a5-4d1b-ad94-d3dc135f225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:27.787145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50067c0d-b1a5-4d1b-ad94-d3dc135f225f None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:27.787425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50067c0d-b1a5-4d1b-ad94-d3dc135f225f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:27.787698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50067c0d-b1a5-4d1b-ad94-d3dc135f225f None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:27.788403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50067c0d-b1a5-4d1b-ad94-d3dc135f225f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:27.789862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50067c0d-b1a5-4d1b-ad94-d3dc135f225f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:27.877893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 549/1110] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:18:27 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:18:30.094232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3ff4-422c-4a0a-967c-76ebd1a17d94 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:30.094748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3ff4-422c-4a0a-967c-76ebd1a17d94 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:30.094867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec3ff4-422c-4a0a-967c-76ebd1a17d94 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:30.096146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 562/1111] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:30.102246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee552d6-0a2b-49b6-b034-03594947d03c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:30.103034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee552d6-0a2b-49b6-b034-03594947d03c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:30.103034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee552d6-0a2b-49b6-b034-03594947d03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:30.119922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5ee552d6-0a2b-49b6-b034-03594947d03c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:30.121719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5ee552d6-0a2b-49b6-b034-03594947d03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:30.187475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 550/1112] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:30.210086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3ff4-422c-4a0a-967c-76ebd1a17d94 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:30.222633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:30.309169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec3ff4-422c-4a0a-967c-76ebd1a17d94 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:30.342200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556513d2-270f-42e7-ab9f-0ba67d837547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:30.343026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556513d2-270f-42e7-ab9f-0ba67d837547 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:30.343315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556513d2-270f-42e7-ab9f-0ba67d837547 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:30.343592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556513d2-270f-42e7-ab9f-0ba67d837547 None admin] PATH_INFO: `/v3/roles/fc5d6a08fc2048f19c14f691e35cd41c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:30.345036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556513d2-270f-42e7-ab9f-0ba67d837547 None admin] RBAC: Authorizing `identity:get_role(role_id=fc5d6a08fc2048f19c14f691e35cd41c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:30.346424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556513d2-270f-42e7-ab9f-0ba67d837547 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:30.348021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 563/1113] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:30 2022] GET /identity/v3/roles/fc5d6a08fc2048f19c14f691e35cd41c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:30.353751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee552d6-0a2b-49b6-b034-03594947d03c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:30.419531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:30.447535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee552d6-0a2b-49b6-b034-03594947d03c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:30.479822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9415721-f02d-4ce2-8562-123558689661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:30.480667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9415721-f02d-4ce2-8562-123558689661 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:30.480946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9415721-f02d-4ce2-8562-123558689661 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:30.481252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9415721-f02d-4ce2-8562-123558689661 None admin] PATH_INFO: `/v3/users/2d2ffa03d7a247b8916daa32f7e09c68` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:30.494329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9415721-f02d-4ce2-8562-123558689661 None admin] RBAC: Authorizing `identity:get_user(user_id=2d2ffa03d7a247b8916daa32f7e09c68)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:30.496285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9415721-f02d-4ce2-8562-123558689661 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:30.497918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 551/1114] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:30 2022] GET /identity/v3/users/2d2ffa03d7a247b8916daa32f7e09c68 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:30.503496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556513d2-270f-42e7-ab9f-0ba67d837547 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:30.579103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:30.599580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556513d2-270f-42e7-ab9f-0ba67d837547 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:30.632964 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4628-72f2-4970-bd09-9739daf5a040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:30.633645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4628-72f2-4970-bd09-9739daf5a040 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:30.633930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4628-72f2-4970-bd09-9739daf5a040 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:30.634206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb4628-72f2-4970-bd09-9739daf5a040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:30.698539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb4628-72f2-4970-bd09-9739daf5a040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:30.700415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb4628-72f2-4970-bd09-9739daf5a040 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:30.773320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:30.794506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 564/1115] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:18:30.800246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9415721-f02d-4ce2-8562-123558689661 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:30.894520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9415721-f02d-4ce2-8562-123558689661 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:30.922947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:30.928556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0bde7-3f68-484e-bbc2-848a943ede60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:30.929434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0bde7-3f68-484e-bbc2-848a943ede60 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:30.929716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0bde7-3f68-484e-bbc2-848a943ede60 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:30.929989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0bde7-3f68-484e-bbc2-848a943ede60 None admin] PATH_INFO: `/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:30.931179 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0bde7-3f68-484e-bbc2-848a943ede60 None admin] RBAC: Authorizing `identity:get_project(project_id=a83cc4f995d44b269ce2d3d59076c4f1)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:30.932609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0bde7-3f68-484e-bbc2-848a943ede60 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:30.934882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 552/1116] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:30 2022] GET /identity/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:30.940794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4628-72f2-4970-bd09-9739daf5a040 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:31.035406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb4628-72f2-4970-bd09-9739daf5a040 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:31.068340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ccadc-5e41-42b8-9695-dfd30d784fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:31.068973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ccadc-5e41-42b8-9695-dfd30d784fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:31.069299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ccadc-5e41-42b8-9695-dfd30d784fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:31.069575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ccadc-5e41-42b8-9695-dfd30d784fe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:31.070390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ccadc-5e41-42b8-9695-dfd30d784fe8 None admin] RBAC: Adding query filter params (role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=a83cc4f995d44b269ce2d3d59076c4f1, user.id=2d2ffa03d7a247b8916daa32f7e09c68) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:31.070698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ccadc-5e41-42b8-9695-dfd30d784fe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:31.072106 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ccadc-5e41-42b8-9695-dfd30d784fe8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:31.077420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 565/1117] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:30 2022] GET /identity/v3/role_assignments?user.id=2d2ffa03d7a247b8916daa32f7e09c68&scope.project.id=a83cc4f995d44b269ce2d3d59076c4f1&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:33.209275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59992138-5b21-4c12-be1c-c3594a6489cb None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:33.209780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59992138-5b21-4c12-be1c-c3594a6489cb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:33.209780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59992138-5b21-4c12-be1c-c3594a6489cb None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:33.211096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 553/1118] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:33.217457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696c583-8f88-48cc-a05c-9a772427634a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:33.218158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696c583-8f88-48cc-a05c-9a772427634a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:33.218158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696c583-8f88-48cc-a05c-9a772427634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:33.235643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8696c583-8f88-48cc-a05c-9a772427634a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:33.236827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8696c583-8f88-48cc-a05c-9a772427634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:33.270999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:33.301476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 566/1119] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:33.324042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59992138-5b21-4c12-be1c-c3594a6489cb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:33.420563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59992138-5b21-4c12-be1c-c3594a6489cb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:33.457866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dfea5-9399-4e8d-9f5a-9f3885628dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:33.458714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dfea5-9399-4e8d-9f5a-9f3885628dfa None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:33.458993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dfea5-9399-4e8d-9f5a-9f3885628dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:33.459267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dfea5-9399-4e8d-9f5a-9f3885628dfa None admin] PATH_INFO: `/v3/roles/fc5d6a08fc2048f19c14f691e35cd41c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:33.460767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dfea5-9399-4e8d-9f5a-9f3885628dfa None admin] RBAC: Authorizing `identity:get_role(role_id=fc5d6a08fc2048f19c14f691e35cd41c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:33.462249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dfea5-9399-4e8d-9f5a-9f3885628dfa None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:33.463601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 554/1120] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:33 2022] GET /identity/v3/roles/fc5d6a08fc2048f19c14f691e35cd41c => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:33.469295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696c583-8f88-48cc-a05c-9a772427634a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:33.563909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696c583-8f88-48cc-a05c-9a772427634a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:33.598329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef728e88-45d5-4ef5-a4fc-f9a32e25c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:33.599190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef728e88-45d5-4ef5-a4fc-f9a32e25c56e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:33.599471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef728e88-45d5-4ef5-a4fc-f9a32e25c56e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:33.599744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef728e88-45d5-4ef5-a4fc-f9a32e25c56e None admin] PATH_INFO: `/v3/users/2d2ffa03d7a247b8916daa32f7e09c68` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:33.600912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef728e88-45d5-4ef5-a4fc-f9a32e25c56e None admin] RBAC: Authorizing `identity:get_user(user_id=2d2ffa03d7a247b8916daa32f7e09c68)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:33.602373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef728e88-45d5-4ef5-a4fc-f9a32e25c56e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:33.603990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 567/1121] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:33 2022] GET /identity/v3/users/2d2ffa03d7a247b8916daa32f7e09c68 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:33.610003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dfea5-9399-4e8d-9f5a-9f3885628dfa None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:33.673798 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:33.707767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dfea5-9399-4e8d-9f5a-9f3885628dfa None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:33.740130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a9de7-5fa9-46b2-bc17-2c1dc51c3693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:33.740917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a9de7-5fa9-46b2-bc17-2c1dc51c3693 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:33.741237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a9de7-5fa9-46b2-bc17-2c1dc51c3693 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:33.741518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a9de7-5fa9-46b2-bc17-2c1dc51c3693 None admin] PATH_INFO: `/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:33.742677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a9de7-5fa9-46b2-bc17-2c1dc51c3693 None admin] RBAC: Authorizing `identity:get_project(project_id=a83cc4f995d44b269ce2d3d59076c4f1)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:33.744133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a9de7-5fa9-46b2-bc17-2c1dc51c3693 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:33.746367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 555/1122] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:33 2022] GET /identity/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:33.752161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef728e88-45d5-4ef5-a4fc-f9a32e25c56e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:33.793339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:33.848758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef728e88-45d5-4ef5-a4fc-f9a32e25c56e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:33.881877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ee6c6-0053-4383-874e-07f46cefb40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:33.882746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee6c6-0053-4383-874e-07f46cefb40a None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:33.883029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee6c6-0053-4383-874e-07f46cefb40a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:33.883308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee6c6-0053-4383-874e-07f46cefb40a None admin] PATH_INFO: `/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1/users/2d2ffa03d7a247b8916daa32f7e09c68/roles/fc5d6a08fc2048f19c14f691e35cd41c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:33.885494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ee6c6-0053-4383-874e-07f46cefb40a None admin] RBAC: Authorizing `identity:create_grant(project_id=a83cc4f995d44b269ce2d3d59076c4f1, user_id=2d2ffa03d7a247b8916daa32f7e09c68, role_id=fc5d6a08fc2048f19c14f691e35cd41c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:33.886947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ee6c6-0053-4383-874e-07f46cefb40a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:33.958134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 568/1123] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:18:33 2022] PUT /identity/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1/users/2d2ffa03d7a247b8916daa32f7e09c68/roles/fc5d6a08fc2048f19c14f691e35cd41c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:18:36.097851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6892745-e4fb-4dbb-a65f-978107566b87 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:36.098403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6892745-e4fb-4dbb-a65f-978107566b87 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:36.098403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6892745-e4fb-4dbb-a65f-978107566b87 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:36.099738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 556/1124] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:36.104847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed873c1-9b72-4ba4-8f53-812662e68b51 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:36.105269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed873c1-9b72-4ba4-8f53-812662e68b51 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:36.105449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed873c1-9b72-4ba4-8f53-812662e68b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:36.121729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4ed873c1-9b72-4ba4-8f53-812662e68b51 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:36.122900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4ed873c1-9b72-4ba4-8f53-812662e68b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:36.186613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 569/1125] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:36.210376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6892745-e4fb-4dbb-a65f-978107566b87 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:36.308449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6892745-e4fb-4dbb-a65f-978107566b87 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:36.341498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3d7b0-0da1-4ede-8cdb-1a4d1a4a6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:36.342343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3d7b0-0da1-4ede-8cdb-1a4d1a4a6a84 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:36.342625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3d7b0-0da1-4ede-8cdb-1a4d1a4a6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:36.342895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3d7b0-0da1-4ede-8cdb-1a4d1a4a6a84 None admin] PATH_INFO: `/v3/roles/fc5d6a08fc2048f19c14f691e35cd41c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:36.344346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3d7b0-0da1-4ede-8cdb-1a4d1a4a6a84 None admin] RBAC: Authorizing `identity:get_role(role_id=fc5d6a08fc2048f19c14f691e35cd41c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:36.345753 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3d7b0-0da1-4ede-8cdb-1a4d1a4a6a84 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:36.347099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 557/1126] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:36 2022] GET /identity/v3/roles/fc5d6a08fc2048f19c14f691e35cd41c => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:36.353079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed873c1-9b72-4ba4-8f53-812662e68b51 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:36.381727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:36.449368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed873c1-9b72-4ba4-8f53-812662e68b51 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:36.481631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3310-d1f7-4b94-9676-7c3917631e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:36.482483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3310-d1f7-4b94-9676-7c3917631e2e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:36.482768 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3310-d1f7-4b94-9676-7c3917631e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:36.483047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3310-d1f7-4b94-9676-7c3917631e2e None admin] PATH_INFO: `/v3/users/2d2ffa03d7a247b8916daa32f7e09c68` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:36.484214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac3310-d1f7-4b94-9676-7c3917631e2e None admin] RBAC: Authorizing `identity:get_user(user_id=2d2ffa03d7a247b8916daa32f7e09c68)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:36.485661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac3310-d1f7-4b94-9676-7c3917631e2e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:36.487262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 570/1127] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:36 2022] GET /identity/v3/users/2d2ffa03d7a247b8916daa32f7e09c68 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:36.492941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3d7b0-0da1-4ede-8cdb-1a4d1a4a6a84 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:36.496304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:36.593809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3d7b0-0da1-4ede-8cdb-1a4d1a4a6a84 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:36.625891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7706707-d295-4f61-b1f2-91a4472fefc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:36.626526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7706707-d295-4f61-b1f2-91a4472fefc4 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:36.626807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7706707-d295-4f61-b1f2-91a4472fefc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:36.627095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7706707-d295-4f61-b1f2-91a4472fefc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:36.690584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7706707-d295-4f61-b1f2-91a4472fefc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:36.692031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7706707-d295-4f61-b1f2-91a4472fefc4 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:36.693814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:36.785847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 558/1128] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:18:36.791806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3310-d1f7-4b94-9676-7c3917631e2e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:36.888961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:36.892626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3310-d1f7-4b94-9676-7c3917631e2e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:36.926244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd1761-888e-4ebe-8cb9-56cc0e21a2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:36.927098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1761-888e-4ebe-8cb9-56cc0e21a2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:36.927379 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1761-888e-4ebe-8cb9-56cc0e21a2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:36.927651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1761-888e-4ebe-8cb9-56cc0e21a2c9 None admin] PATH_INFO: `/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:36.928804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd1761-888e-4ebe-8cb9-56cc0e21a2c9 None admin] RBAC: Authorizing `identity:get_project(project_id=a83cc4f995d44b269ce2d3d59076c4f1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:36.930258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd1761-888e-4ebe-8cb9-56cc0e21a2c9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:36.931934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 571/1129] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:36 2022] GET /identity/v3/projects/a83cc4f995d44b269ce2d3d59076c4f1 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:36.937982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7706707-d295-4f61-b1f2-91a4472fefc4 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:37.032016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7706707-d295-4f61-b1f2-91a4472fefc4 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:37.033509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:37.064763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb8663-3558-4727-8ebd-d61cefbd5b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:37.065364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb8663-3558-4727-8ebd-d61cefbd5b11 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:37.065647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb8663-3558-4727-8ebd-d61cefbd5b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:37.065922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb8663-3558-4727-8ebd-d61cefbd5b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:37.066736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb8663-3558-4727-8ebd-d61cefbd5b11 None admin] RBAC: Adding query filter params (role.id=fc5d6a08fc2048f19c14f691e35cd41c, scope.project.id=a83cc4f995d44b269ce2d3d59076c4f1, user.id=2d2ffa03d7a247b8916daa32f7e09c68) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:37.067044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb8663-3558-4727-8ebd-d61cefbd5b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:37.068479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb8663-3558-4727-8ebd-d61cefbd5b11 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:37.074643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 559/1130] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:36 2022] GET /identity/v3/role_assignments?user.id=2d2ffa03d7a247b8916daa32f7e09c68&scope.project.id=a83cc4f995d44b269ce2d3d59076c4f1&role.id=fc5d6a08fc2048f19c14f691e35cd41c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:39.346898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6561e-dc73-4426-b811-782c4500cd34 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:39.347420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6561e-dc73-4426-b811-782c4500cd34 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:39.347420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6561e-dc73-4426-b811-782c4500cd34 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:39.349035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 572/1131] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:39.354409 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb21c2-7931-483f-bbff-65f1b74d0faa None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:39.354848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb21c2-7931-483f-bbff-65f1b74d0faa None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:39.354967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb21c2-7931-483f-bbff-65f1b74d0faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:39.371085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-69cb21c2-7931-483f-bbff-65f1b74d0faa None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:39.372255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-69cb21c2-7931-483f-bbff-65f1b74d0faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:39.435661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 560/1132] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:39.464981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6561e-dc73-4426-b811-782c4500cd34 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:39.566439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6561e-dc73-4426-b811-782c4500cd34 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:39.599181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fbad3-85d7-46cb-be84-8a9eaced1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:39.599894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fbad3-85d7-46cb-be84-8a9eaced1f01 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:39.600175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fbad3-85d7-46cb-be84-8a9eaced1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:39.600449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fbad3-85d7-46cb-be84-8a9eaced1f01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:39.601628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fbad3-85d7-46cb-be84-8a9eaced1f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:39.603050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fbad3-85d7-46cb-be84-8a9eaced1f01 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:39.604683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 573/1133] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:39.610517 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb21c2-7931-483f-bbff-65f1b74d0faa None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:39.698070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:39.703886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb21c2-7931-483f-bbff-65f1b74d0faa None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:39.736923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bb2b67-52e7-4f42-b56e-e39661e3d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:39.737595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb2b67-52e7-4f42-b56e-e39661e3d4cf None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:39.737876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb2b67-52e7-4f42-b56e-e39661e3d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:39.738150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb2b67-52e7-4f42-b56e-e39661e3d4cf None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:39.738847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb2b67-52e7-4f42-b56e-e39661e3d4cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:39.740260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bb2b67-52e7-4f42-b56e-e39661e3d4cf None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:39.813057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 561/1134] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:18:39 2022] POST /identity/v3/projects => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:42.045486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8ec1-51fe-4cfe-82cf-077ff88ed0f5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:42.046015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8ec1-51fe-4cfe-82cf-077ff88ed0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:42.046015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a8ec1-51fe-4cfe-82cf-077ff88ed0f5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:42.047393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 574/1135] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:42.052517 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f99905-1735-48c3-a3c2-e4daeba6d6d3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:42.052885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f99905-1735-48c3-a3c2-e4daeba6d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:42.053085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f99905-1735-48c3-a3c2-e4daeba6d6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:42.070088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-34f99905-1735-48c3-a3c2-e4daeba6d6d3 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:42.071266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-34f99905-1735-48c3-a3c2-e4daeba6d6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:42.135201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 562/1136] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:42.159689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a8ec1-51fe-4cfe-82cf-077ff88ed0f5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:42.194744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:42.258415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a8ec1-51fe-4cfe-82cf-077ff88ed0f5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:42.290925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf063f92-8f00-46fc-b233-11b788c41315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:42.291709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063f92-8f00-46fc-b233-11b788c41315 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:42.292498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063f92-8f00-46fc-b233-11b788c41315 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:42.292770 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063f92-8f00-46fc-b233-11b788c41315 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:42.294033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf063f92-8f00-46fc-b233-11b788c41315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:42.295448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf063f92-8f00-46fc-b233-11b788c41315 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:42.297040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 575/1137] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:18:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:42.302707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f99905-1735-48c3-a3c2-e4daeba6d6d3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:42.376534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:42.396320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f99905-1735-48c3-a3c2-e4daeba6d6d3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:42.429244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dc863-05d3-43ed-a543-18a9f62b49e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:42.429840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dc863-05d3-43ed-a543-18a9f62b49e0 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:42.430122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dc863-05d3-43ed-a543-18a9f62b49e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:42.430396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dc863-05d3-43ed-a543-18a9f62b49e0 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:42.431092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dc863-05d3-43ed-a543-18a9f62b49e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:42.432532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dc863-05d3-43ed-a543-18a9f62b49e0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:42.508862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 563/1138] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:18:42 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:44.697566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cff7f4-80ba-430c-8969-18f437a72533 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:44.698148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cff7f4-80ba-430c-8969-18f437a72533 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:44.698148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cff7f4-80ba-430c-8969-18f437a72533 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:44.700052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 576/1139] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:44.705312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86986f80-bcc6-42fc-be27-704345483cd8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:44.705696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86986f80-bcc6-42fc-be27-704345483cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:44.705876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86986f80-bcc6-42fc-be27-704345483cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:44.722187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-86986f80-bcc6-42fc-be27-704345483cd8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:44.723623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-86986f80-bcc6-42fc-be27-704345483cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:44.787049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 564/1140] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:44.810003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cff7f4-80ba-430c-8969-18f437a72533 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:44.916701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cff7f4-80ba-430c-8969-18f437a72533 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:44.916701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:44.942490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386ee1f-65e2-43b0-8fd8-e259ce86a555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:44.943259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386ee1f-65e2-43b0-8fd8-e259ce86a555 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:44.943545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386ee1f-65e2-43b0-8fd8-e259ce86a555 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:44.943818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386ee1f-65e2-43b0-8fd8-e259ce86a555 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:44.954302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386ee1f-65e2-43b0-8fd8-e259ce86a555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b386ee1f-65e2-43b0-8fd8-e259ce86a555 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:44.961877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:44.963001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:44.964088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 577/1141] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:18:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:44.969224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86986f80-bcc6-42fc-be27-704345483cd8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:45.028708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:45.064923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86986f80-bcc6-42fc-be27-704345483cd8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:45.098276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:45.098926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:45.099209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:45.099485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:45.100213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:45.100522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:45.101932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:45.110221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 565/1142] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:18:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:45.115743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386ee1f-65e2-43b0-8fd8-e259ce86a555 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:45.216062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386ee1f-65e2-43b0-8fd8-e259ce86a555 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:45.249423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09af74-b251-4f88-baec-64a98030905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:45.250275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09af74-b251-4f88-baec-64a98030905c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:45.250553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09af74-b251-4f88-baec-64a98030905c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:45.250821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09af74-b251-4f88-baec-64a98030905c None admin] PATH_INFO: `/v3/users/3c19d1d44fd9488886fdde365437f56a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:45.263324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09af74-b251-4f88-baec-64a98030905c None admin] RBAC: Authorizing `identity:get_user(user_id=3c19d1d44fd9488886fdde365437f56a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:45.264746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09af74-b251-4f88-baec-64a98030905c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:45.266401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 578/1143] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:45 2022] GET /identity/v3/users/3c19d1d44fd9488886fdde365437f56a => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:45.272082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:45.367212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655f14d-3315-447d-a680-7ae7d6a5f070 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:45.391316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:45.399194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9bf0af-4a57-46e8-9d7b-17f9621785c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:45.402257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bf0af-4a57-46e8-9d7b-17f9621785c2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:45.402257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bf0af-4a57-46e8-9d7b-17f9621785c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:45.402257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bf0af-4a57-46e8-9d7b-17f9621785c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:45.464679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9bf0af-4a57-46e8-9d7b-17f9621785c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:45.466180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9bf0af-4a57-46e8-9d7b-17f9621785c2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:45.563491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 566/1144] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:18:45.569365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09af74-b251-4f88-baec-64a98030905c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:45.590742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:45.665509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09af74-b251-4f88-baec-64a98030905c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:45.698028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bba95a-3f5d-4993-b844-865ec32cb262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:45.698904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bba95a-3f5d-4993-b844-865ec32cb262 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:45.699185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bba95a-3f5d-4993-b844-865ec32cb262 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:45.699456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bba95a-3f5d-4993-b844-865ec32cb262 None admin] PATH_INFO: `/v3/projects/4951f9f421d54a71a82b62cdc54f2f61` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:45.700629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bba95a-3f5d-4993-b844-865ec32cb262 None admin] RBAC: Authorizing `identity:get_project(project_id=4951f9f421d54a71a82b62cdc54f2f61)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:45.702077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bba95a-3f5d-4993-b844-865ec32cb262 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:45.703745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 579/1145] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:45 2022] GET /identity/v3/projects/4951f9f421d54a71a82b62cdc54f2f61 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:45.709784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9bf0af-4a57-46e8-9d7b-17f9621785c2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:45.736392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:45.804428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9bf0af-4a57-46e8-9d7b-17f9621785c2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:45.836389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a8dc-9977-4279-9a1a-ba1128e20d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:45.837016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a8dc-9977-4279-9a1a-ba1128e20d3f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:45.837326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a8dc-9977-4279-9a1a-ba1128e20d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:45.837600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a8dc-9977-4279-9a1a-ba1128e20d3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:45.838406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a8dc-9977-4279-9a1a-ba1128e20d3f None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=4951f9f421d54a71a82b62cdc54f2f61, user.id=3c19d1d44fd9488886fdde365437f56a) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:45.838775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a8dc-9977-4279-9a1a-ba1128e20d3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:45.840690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a8dc-9977-4279-9a1a-ba1128e20d3f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:45.845901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 567/1146] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:45 2022] GET /identity/v3/role_assignments?user.id=3c19d1d44fd9488886fdde365437f56a&scope.project.id=4951f9f421d54a71a82b62cdc54f2f61&role.id=bb0756f8af694633b9743a958eb46e7a => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:47.993328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b4888-7cbd-485c-b7b0-823768cfd5e2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:47.993823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b4888-7cbd-485c-b7b0-823768cfd5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:47.993914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b4888-7cbd-485c-b7b0-823768cfd5e2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:47.995246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 580/1147] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:48.000457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d3d23-a84c-4c83-baba-edfda5bcca8a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:48.000889 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d3d23-a84c-4c83-baba-edfda5bcca8a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:48.001016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d3d23-a84c-4c83-baba-edfda5bcca8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:48.017382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8f1d3d23-a84c-4c83-baba-edfda5bcca8a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:48.018554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8f1d3d23-a84c-4c83-baba-edfda5bcca8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:48.082622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 568/1148] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:48.105126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b4888-7cbd-485c-b7b0-823768cfd5e2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:48.189228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:48.201682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b4888-7cbd-485c-b7b0-823768cfd5e2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:48.235361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501168b-f067-4ad1-8a31-ed0ae34e3dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:48.236137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501168b-f067-4ad1-8a31-ed0ae34e3dda None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:48.236415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501168b-f067-4ad1-8a31-ed0ae34e3dda None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:48.236684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501168b-f067-4ad1-8a31-ed0ae34e3dda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:48.246520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501168b-f067-4ad1-8a31-ed0ae34e3dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4501168b-f067-4ad1-8a31-ed0ae34e3dda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:48.254063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:48.255235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:48.256438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 581/1149] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:48.261684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d3d23-a84c-4c83-baba-edfda5bcca8a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:48.282317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:48.375069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d3d23-a84c-4c83-baba-edfda5bcca8a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:48.415478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:48.416141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:48.416424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:48.416699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:48.417468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:48.417780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:48.419149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:48.426929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 569/1150] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:48.433410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501168b-f067-4ad1-8a31-ed0ae34e3dda None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:48.529519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501168b-f067-4ad1-8a31-ed0ae34e3dda None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:48.561912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9262-7df5-46bc-9709-a00cf0848c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:48.562753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9262-7df5-46bc-9709-a00cf0848c05 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:48.563036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9262-7df5-46bc-9709-a00cf0848c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:48.563313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac9262-7df5-46bc-9709-a00cf0848c05 None admin] PATH_INFO: `/v3/users/3c19d1d44fd9488886fdde365437f56a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:48.564466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac9262-7df5-46bc-9709-a00cf0848c05 None admin] RBAC: Authorizing `identity:get_user(user_id=3c19d1d44fd9488886fdde365437f56a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:48.565920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac9262-7df5-46bc-9709-a00cf0848c05 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:48.567504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 582/1151] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:48 2022] GET /identity/v3/users/3c19d1d44fd9488886fdde365437f56a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:48.573629 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:48.651752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:48.668395 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d4495-f622-4e28-82f5-b2fbf3280991 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:48.702722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b20f0-abf5-4aa1-b9cc-b7dbe2add5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:48.703576 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b20f0-abf5-4aa1-b9cc-b7dbe2add5da None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:48.703854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b20f0-abf5-4aa1-b9cc-b7dbe2add5da None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:48.704125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b20f0-abf5-4aa1-b9cc-b7dbe2add5da None admin] PATH_INFO: `/v3/projects/4951f9f421d54a71a82b62cdc54f2f61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:48.705312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b20f0-abf5-4aa1-b9cc-b7dbe2add5da None admin] RBAC: Authorizing `identity:get_project(project_id=4951f9f421d54a71a82b62cdc54f2f61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:48.706762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b20f0-abf5-4aa1-b9cc-b7dbe2add5da None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:48.708435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 570/1152] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:48 2022] GET /identity/v3/projects/4951f9f421d54a71a82b62cdc54f2f61 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:48.714385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9262-7df5-46bc-9709-a00cf0848c05 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:48.733172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:48.817285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac9262-7df5-46bc-9709-a00cf0848c05 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:48.848846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4a0cf-f63b-4e6c-b4ed-805072657779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:48.849738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4a0cf-f63b-4e6c-b4ed-805072657779 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:48.850017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4a0cf-f63b-4e6c-b4ed-805072657779 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:48.850289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4a0cf-f63b-4e6c-b4ed-805072657779 None admin] PATH_INFO: `/v3/projects/4951f9f421d54a71a82b62cdc54f2f61/users/3c19d1d44fd9488886fdde365437f56a/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:48.852195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4a0cf-f63b-4e6c-b4ed-805072657779 None admin] RBAC: Authorizing `identity:create_grant(project_id=4951f9f421d54a71a82b62cdc54f2f61, user_id=3c19d1d44fd9488886fdde365437f56a, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:48.853636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4a0cf-f63b-4e6c-b4ed-805072657779 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:48.946708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 583/1153] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:18:48 2022] PUT /identity/v3/projects/4951f9f421d54a71a82b62cdc54f2f61/users/3c19d1d44fd9488886fdde365437f56a/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:18:51.090483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0942e-7b1e-4b32-9b88-3dc9097d933b None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:51.091031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0942e-7b1e-4b32-9b88-3dc9097d933b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:51.091031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0942e-7b1e-4b32-9b88-3dc9097d933b None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:51.092373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 571/1154] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:51.097889 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a567f-a500-4256-aba4-3524babd3684 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:51.098268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a567f-a500-4256-aba4-3524babd3684 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:51.098441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a567f-a500-4256-aba4-3524babd3684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:51.114734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-962a567f-a500-4256-aba4-3524babd3684 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:51.115895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-962a567f-a500-4256-aba4-3524babd3684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:51.179428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 584/1155] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:18:51.202247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0942e-7b1e-4b32-9b88-3dc9097d933b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:51.300039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab0942e-7b1e-4b32-9b88-3dc9097d933b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:51.334539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a7a7a-d3e6-4ff6-8320-bb1dc5c41fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:51.335317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a7a7a-d3e6-4ff6-8320-bb1dc5c41fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:51.335603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a7a7a-d3e6-4ff6-8320-bb1dc5c41fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:51.335877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a7a7a-d3e6-4ff6-8320-bb1dc5c41fa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:51.346347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a7a7a-d3e6-4ff6-8320-bb1dc5c41fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1e3a7a7a-d3e6-4ff6-8320-bb1dc5c41fa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:18:51.354122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:51.355210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:18:51.356452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 572/1156] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:51.361337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a567f-a500-4256-aba4-3524babd3684 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:51.369312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:51.456709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a567f-a500-4256-aba4-3524babd3684 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:51.488810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:51.489489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:51.489773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:51.490049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:51.490794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:51.491102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:51.492460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:51.500764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 585/1157] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:18:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:51.506491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a7a7a-d3e6-4ff6-8320-bb1dc5c41fa3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:51.519387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:51.607883 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a7a7a-d3e6-4ff6-8320-bb1dc5c41fa3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:51.640208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d00a15-1404-4850-8953-21f988e19378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:51.641058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00a15-1404-4850-8953-21f988e19378 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:51.641366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00a15-1404-4850-8953-21f988e19378 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:51.641640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d00a15-1404-4850-8953-21f988e19378 None admin] PATH_INFO: `/v3/users/3c19d1d44fd9488886fdde365437f56a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:51.642796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d00a15-1404-4850-8953-21f988e19378 None admin] RBAC: Authorizing `identity:get_user(user_id=3c19d1d44fd9488886fdde365437f56a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:51.644217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d00a15-1404-4850-8953-21f988e19378 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:51.645848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 573/1158] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:18:51 2022] GET /identity/v3/users/3c19d1d44fd9488886fdde365437f56a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:51.651502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:51.717321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:51.749682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709ff90-3787-4b4b-ac77-864437dc6d19 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:51.782197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faece106-2083-4eb0-aee3-e3493fd23965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:51.782832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faece106-2083-4eb0-aee3-e3493fd23965 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:51.783115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faece106-2083-4eb0-aee3-e3493fd23965 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:51.783392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faece106-2083-4eb0-aee3-e3493fd23965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:51.847766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faece106-2083-4eb0-aee3-e3493fd23965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:51.849231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faece106-2083-4eb0-aee3-e3493fd23965 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:51.911569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:51.948444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 586/1159] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:18:51.954559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d00a15-1404-4850-8953-21f988e19378 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:52.029120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:52.049682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d00a15-1404-4850-8953-21f988e19378 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:52.082408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d972e-dbd6-4bb0-ba13-d933e3e23de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:52.083254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d972e-dbd6-4bb0-ba13-d933e3e23de2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:52.083538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d972e-dbd6-4bb0-ba13-d933e3e23de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:52.083814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d972e-dbd6-4bb0-ba13-d933e3e23de2 None admin] PATH_INFO: `/v3/projects/4951f9f421d54a71a82b62cdc54f2f61` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:52.084968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d972e-dbd6-4bb0-ba13-d933e3e23de2 None admin] RBAC: Authorizing `identity:get_project(project_id=4951f9f421d54a71a82b62cdc54f2f61)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:52.086428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d972e-dbd6-4bb0-ba13-d933e3e23de2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:52.088088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 574/1160] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:18:51 2022] GET /identity/v3/projects/4951f9f421d54a71a82b62cdc54f2f61 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:18:52.094209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faece106-2083-4eb0-aee3-e3493fd23965 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:52.189068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faece106-2083-4eb0-aee3-e3493fd23965 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:52.221223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ca4ba-d0b0-4a24-ae85-99f51d74ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:52.221852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca4ba-d0b0-4a24-ae85-99f51d74ba3c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:52.222130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca4ba-d0b0-4a24-ae85-99f51d74ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:52.222402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca4ba-d0b0-4a24-ae85-99f51d74ba3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:52.223465 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ca4ba-d0b0-4a24-ae85-99f51d74ba3c None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=4951f9f421d54a71a82b62cdc54f2f61, user.id=3c19d1d44fd9488886fdde365437f56a) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:52.223771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ca4ba-d0b0-4a24-ae85-99f51d74ba3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:52.225198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ca4ba-d0b0-4a24-ae85-99f51d74ba3c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:52.230807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 587/1161] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:18:52 2022] GET /identity/v3/role_assignments?user.id=3c19d1d44fd9488886fdde365437f56a&scope.project.id=4951f9f421d54a71a82b62cdc54f2f61&role.id=bb0756f8af694633b9743a958eb46e7a => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:54.419561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338c617-1860-4e8f-a579-e9ea0a8b329e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:54.420085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338c617-1860-4e8f-a579-e9ea0a8b329e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:54.420173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338c617-1860-4e8f-a579-e9ea0a8b329e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:54.421529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 575/1162] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:54.426747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd9407-1e22-4fc1-bf40-91717ef2d2a5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:54.427112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd9407-1e22-4fc1-bf40-91717ef2d2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:54.427343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd9407-1e22-4fc1-bf40-91717ef2d2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:54.443559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8ffd9407-1e22-4fc1-bf40-91717ef2d2a5 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:54.444716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8ffd9407-1e22-4fc1-bf40-91717ef2d2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:54.469486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:54.508652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 588/1163] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:54.530479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338c617-1860-4e8f-a579-e9ea0a8b329e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:54.629859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338c617-1860-4e8f-a579-e9ea0a8b329e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:54.662189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb16089-23eb-40d2-94f8-717cb1320801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:54.662825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb16089-23eb-40d2-94f8-717cb1320801 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:54.663110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb16089-23eb-40d2-94f8-717cb1320801 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:54.663384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb16089-23eb-40d2-94f8-717cb1320801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:54.678898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:54.728042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb16089-23eb-40d2-94f8-717cb1320801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:54.730113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb16089-23eb-40d2-94f8-717cb1320801 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:54.821868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 576/1164] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:18:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:18:54.828417 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd9407-1e22-4fc1-bf40-91717ef2d2a5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:54.923397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd9407-1e22-4fc1-bf40-91717ef2d2a5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:54.956225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c2d70-80ae-4c85-8c49-837640b54eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:54.956947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c2d70-80ae-4c85-8c49-837640b54eba None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:54.957260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c2d70-80ae-4c85-8c49-837640b54eba None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:54.957533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c2d70-80ae-4c85-8c49-837640b54eba None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:54.964596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c2d70-80ae-4c85-8c49-837640b54eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:54.966046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c2d70-80ae-4c85-8c49-837640b54eba None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f52c2d70-80ae-4c85-8c49-837640b54eba None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:18:54.980251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:18:54.981441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:18:54.982628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 589/1165] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:18:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:18:54.987396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb16089-23eb-40d2-94f8-717cb1320801 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:55.032186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:55.084276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb16089-23eb-40d2-94f8-717cb1320801 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:55.118432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649410c-4a8e-42e5-87ad-08dbffea160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:55.119087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649410c-4a8e-42e5-87ad-08dbffea160c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:55.119369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649410c-4a8e-42e5-87ad-08dbffea160c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:55.119642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649410c-4a8e-42e5-87ad-08dbffea160c None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:55.120380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649410c-4a8e-42e5-87ad-08dbffea160c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:18:55.120688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649410c-4a8e-42e5-87ad-08dbffea160c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:55.122092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649410c-4a8e-42e5-87ad-08dbffea160c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:55.129061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 577/1166] 10.222.0.41 () {56 vars in 1115 bytes} [Sat May 28 03:18:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:18:57.279254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e05ee-8962-46d2-ad2e-d86e744a57d3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:57.279806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e05ee-8962-46d2-ad2e-d86e744a57d3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:57.279806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e05ee-8962-46d2-ad2e-d86e744a57d3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:57.281211 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 590/1167] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:57.286323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaec40b-9354-4bf4-8da1-9f8aaf1e6626 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:57.286736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaec40b-9354-4bf4-8da1-9f8aaf1e6626 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:57.286883 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaec40b-9354-4bf4-8da1-9f8aaf1e6626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:57.303579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bbaec40b-9354-4bf4-8da1-9f8aaf1e6626 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:57.304769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bbaec40b-9354-4bf4-8da1-9f8aaf1e6626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:57.369877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 578/1168] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:57.392592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e05ee-8962-46d2-ad2e-d86e744a57d3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:57.417631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:18:57.492114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e05ee-8962-46d2-ad2e-d86e744a57d3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:18:57.524888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b79bb-59e0-49e3-aa5e-f54034cf1637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:18:57.525627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b79bb-59e0-49e3-aa5e-f54034cf1637 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:57.525939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b79bb-59e0-49e3-aa5e-f54034cf1637 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:57.526227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b79bb-59e0-49e3-aa5e-f54034cf1637 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:57.526967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b79bb-59e0-49e3-aa5e-f54034cf1637 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:18:57.528322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b79bb-59e0-49e3-aa5e-f54034cf1637 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:18:57.598880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 591/1169] 10.222.0.41 () {60 vars in 1138 bytes} [Sat May 28 03:18:57 2022] POST /identity/v3/domains => generated 267 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:18:59.802125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8022f-d6ef-4bb6-993d-6ff8caa64cff None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:59.802673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8022f-d6ef-4bb6-993d-6ff8caa64cff None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:59.802673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8022f-d6ef-4bb6-993d-6ff8caa64cff None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:59.804047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 579/1170] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:18:59.809779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e06b0-ee77-4fae-a180-eddd9c154207 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:18:59.810172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e06b0-ee77-4fae-a180-eddd9c154207 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:18:59.810343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e06b0-ee77-4fae-a180-eddd9c154207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:18:59.827369 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3f8e06b0-ee77-4fae-a180-eddd9c154207 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:18:59.828551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3f8e06b0-ee77-4fae-a180-eddd9c154207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:18:59.893154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 592/1171] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:18:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:18:59.916162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8022f-d6ef-4bb6-993d-6ff8caa64cff None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:18:59.958089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:00.013955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8022f-d6ef-4bb6-993d-6ff8caa64cff None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:00.046063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4feeb2-5b27-4c75-9668-cc12d910369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:00.046900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4feeb2-5b27-4c75-9668-cc12d910369f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:00.047182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4feeb2-5b27-4c75-9668-cc12d910369f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:00.047453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4feeb2-5b27-4c75-9668-cc12d910369f None admin] PATH_INFO: `/v3/domains/d7cefb973b814c87af344e7b494cbfab` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:00.058073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4feeb2-5b27-4c75-9668-cc12d910369f None admin] RBAC: Authorizing `identity:get_domain(domain_id=d7cefb973b814c87af344e7b494cbfab)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:00.059455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4feeb2-5b27-4c75-9668-cc12d910369f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:00.061027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 580/1172] 10.222.0.41 () {56 vars in 1183 bytes} [Sat May 28 03:18:59 2022] GET /identity/v3/domains/d7cefb973b814c87af344e7b494cbfab => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:00.066715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e06b0-ee77-4fae-a180-eddd9c154207 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:00.080470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:00.163107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e06b0-ee77-4fae-a180-eddd9c154207 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:00.196344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023de067-ebcc-4c3a-8f23-e07654638e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:00.196985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de067-ebcc-4c3a-8f23-e07654638e51 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:00.197303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de067-ebcc-4c3a-8f23-e07654638e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:00.197586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de067-ebcc-4c3a-8f23-e07654638e51 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:00.198308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023de067-ebcc-4c3a-8f23-e07654638e51 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:00.199733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023de067-ebcc-4c3a-8f23-e07654638e51 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:00.310682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 593/1173] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:19:00 2022] POST /identity/v3/projects => generated 361 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:19:02.513547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f797bd-a2a3-46c2-ae39-958300ae4f44 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:02.514089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f797bd-a2a3-46c2-ae39-958300ae4f44 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:02.514089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f797bd-a2a3-46c2-ae39-958300ae4f44 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:02.515450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 581/1174] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:19:02.520565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6beb4-971a-4592-9111-8ac807eb9926 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:02.520986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6beb4-971a-4592-9111-8ac807eb9926 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:02.521166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6beb4-971a-4592-9111-8ac807eb9926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:02.538171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b2b6beb4-971a-4592-9111-8ac807eb9926 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:19:02.539349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b2b6beb4-971a-4592-9111-8ac807eb9926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:19:02.595926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:02.604173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 594/1175] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:19:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:19:02.628184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f797bd-a2a3-46c2-ae39-958300ae4f44 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:02.723674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:02.727330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f797bd-a2a3-46c2-ae39-958300ae4f44 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:02.759417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bd907-545d-41f8-86bd-f3055dc76b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:02.760862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bd907-545d-41f8-86bd-f3055dc76b38 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:02.761369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bd907-545d-41f8-86bd-f3055dc76b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:02.761814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bd907-545d-41f8-86bd-f3055dc76b38 None admin] PATH_INFO: `/v3/domains/d7cefb973b814c87af344e7b494cbfab` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:02.763259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bd907-545d-41f8-86bd-f3055dc76b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d7cefb973b814c87af344e7b494cbfab)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:02.764744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bd907-545d-41f8-86bd-f3055dc76b38 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:02.766305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 582/1176] 10.222.0.41 () {56 vars in 1183 bytes} [Sat May 28 03:19:02 2022] GET /identity/v3/domains/d7cefb973b814c87af344e7b494cbfab => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:02.771853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6beb4-971a-4592-9111-8ac807eb9926 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:02.867000 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6beb4-971a-4592-9111-8ac807eb9926 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:02.899868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba2a3f-2f5d-4059-987d-5a4b327432a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:02.900515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2a3f-2f5d-4059-987d-5a4b327432a6 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:02.900826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2a3f-2f5d-4059-987d-5a4b327432a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:02.901134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba2a3f-2f5d-4059-987d-5a4b327432a6 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:02.901853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba2a3f-2f5d-4059-987d-5a4b327432a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:02.903290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba2a3f-2f5d-4059-987d-5a4b327432a6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:02.972075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 595/1177] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:19:02 2022] POST /identity/v3/users => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:19:05.241586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9f1ef-2452-4ce5-9847-f13e82edf4f1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:05.242096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9f1ef-2452-4ce5-9847-f13e82edf4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:05.242096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9f1ef-2452-4ce5-9847-f13e82edf4f1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:05.243403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 583/1178] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:19:05.248570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7b085-dd33-4d8d-9dce-a1f6d12ddc0e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:05.248941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7b085-dd33-4d8d-9dce-a1f6d12ddc0e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:05.249169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7b085-dd33-4d8d-9dce-a1f6d12ddc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:05.265597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d0d7b085-dd33-4d8d-9dce-a1f6d12ddc0e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:19:05.266786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d0d7b085-dd33-4d8d-9dce-a1f6d12ddc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:19:05.316011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:05.336497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 596/1179] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:19:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:19:05.362822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9f1ef-2452-4ce5-9847-f13e82edf4f1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:05.467398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9f1ef-2452-4ce5-9847-f13e82edf4f1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:05.499678 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7125e-1357-4a0b-9066-02e55afe34a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:05.500458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7125e-1357-4a0b-9066-02e55afe34a6 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:05.500744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7125e-1357-4a0b-9066-02e55afe34a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:05.501022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7125e-1357-4a0b-9066-02e55afe34a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:05.511509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7125e-1357-4a0b-9066-02e55afe34a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-9df7125e-1357-4a0b-9066-02e55afe34a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:19:05.519190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:19:05.520365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:19:05.521526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 584/1180] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:19:05.527032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7b085-dd33-4d8d-9dce-a1f6d12ddc0e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:05.625817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7b085-dd33-4d8d-9dce-a1f6d12ddc0e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:05.659458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:05.660118 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:05.660401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:05.660675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:05.661447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:19:05.661753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:05.663109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:05.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 597/1181] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:05.676912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7125e-1357-4a0b-9066-02e55afe34a6 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:05.705770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:05.773678 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7125e-1357-4a0b-9066-02e55afe34a6 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:05.805312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1990-35a7-4b45-ac7e-858915269320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:05.806158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1990-35a7-4b45-ac7e-858915269320 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:05.806436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1990-35a7-4b45-ac7e-858915269320 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:05.806707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1990-35a7-4b45-ac7e-858915269320 None admin] PATH_INFO: `/v3/users/55f918327e3d4d87809b144c2f30605c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:05.819717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1990-35a7-4b45-ac7e-858915269320 None admin] RBAC: Authorizing `identity:get_user(user_id=55f918327e3d4d87809b144c2f30605c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:05.821197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1990-35a7-4b45-ac7e-858915269320 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:05.823342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 585/1182] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:19:05 2022] GET /identity/v3/users/55f918327e3d4d87809b144c2f30605c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:05.829022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:05.870993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:05.925759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa2d6-9dfc-4e08-ad96-2dbeab7c8978 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:05.958895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609d29d-29e2-417a-b4fb-f7e20282a0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:05.959527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609d29d-29e2-417a-b4fb-f7e20282a0ed None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:05.959842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609d29d-29e2-417a-b4fb-f7e20282a0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:05.960115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609d29d-29e2-417a-b4fb-f7e20282a0ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:06.026275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609d29d-29e2-417a-b4fb-f7e20282a0ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:06.027652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609d29d-29e2-417a-b4fb-f7e20282a0ed None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:06.067947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:06.123425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 598/1183] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:19:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:19:06.129419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1990-35a7-4b45-ac7e-858915269320 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:06.208989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:06.223162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1990-35a7-4b45-ac7e-858915269320 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:06.256050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7f9b7-b6af-4048-aaa8-b2354552a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:06.256893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f9b7-b6af-4048-aaa8-b2354552a785 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:06.257198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f9b7-b6af-4048-aaa8-b2354552a785 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:06.257468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7f9b7-b6af-4048-aaa8-b2354552a785 None admin] PATH_INFO: `/v3/projects/63a4ab5043b94a3c9a5205789cb7497b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:06.258641 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7f9b7-b6af-4048-aaa8-b2354552a785 None admin] RBAC: Authorizing `identity:get_project(project_id=63a4ab5043b94a3c9a5205789cb7497b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:06.260060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7f9b7-b6af-4048-aaa8-b2354552a785 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:06.261732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 586/1184] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:19:06 2022] GET /identity/v3/projects/63a4ab5043b94a3c9a5205789cb7497b => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:06.267638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609d29d-29e2-417a-b4fb-f7e20282a0ed None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:06.363079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609d29d-29e2-417a-b4fb-f7e20282a0ed None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:06.395599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08678e05-fbff-495b-8080-523307b917ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:06.396239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08678e05-fbff-495b-8080-523307b917ee None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:06.396524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08678e05-fbff-495b-8080-523307b917ee None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:06.396801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08678e05-fbff-495b-8080-523307b917ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:06.397655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08678e05-fbff-495b-8080-523307b917ee None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=63a4ab5043b94a3c9a5205789cb7497b, user.id=55f918327e3d4d87809b144c2f30605c) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:19:06.397970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08678e05-fbff-495b-8080-523307b917ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:06.399379 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08678e05-fbff-495b-8080-523307b917ee None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:06.405236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 599/1185] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:19:06 2022] GET /identity/v3/role_assignments?user.id=55f918327e3d4d87809b144c2f30605c&scope.project.id=63a4ab5043b94a3c9a5205789cb7497b&role.id=bb0756f8af694633b9743a958eb46e7a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:08.598518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3f415-5b80-46af-9f08-bfbe6bc5e205 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:08.599048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3f415-5b80-46af-9f08-bfbe6bc5e205 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:08.599164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3f415-5b80-46af-9f08-bfbe6bc5e205 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:08.600443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 587/1186] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:19:08.605746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8cc08-77ea-4e50-b611-81c743c404a7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:08.606149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8cc08-77ea-4e50-b611-81c743c404a7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:08.606297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8cc08-77ea-4e50-b611-81c743c404a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:08.617125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:08.622869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4cf8cc08-77ea-4e50-b611-81c743c404a7 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:19:08.624042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4cf8cc08-77ea-4e50-b611-81c743c404a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:19:08.693560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 600/1187] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:19:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:19:08.716207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3f415-5b80-46af-9f08-bfbe6bc5e205 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:08.815947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3f415-5b80-46af-9f08-bfbe6bc5e205 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:08.849236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ae18-6718-48d9-a8e4-dc9fa8fa0019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:08.849957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ae18-6718-48d9-a8e4-dc9fa8fa0019 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:08.850243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ae18-6718-48d9-a8e4-dc9fa8fa0019 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:08.850521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634ae18-6718-48d9-a8e4-dc9fa8fa0019 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:08.860591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634ae18-6718-48d9-a8e4-dc9fa8fa0019 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1634ae18-6718-48d9-a8e4-dc9fa8fa0019 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:19:08.868193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:19:08.869837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:19:08.872021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 588/1188] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:19:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:19:08.875863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8cc08-77ea-4e50-b611-81c743c404a7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:08.971991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8cc08-77ea-4e50-b611-81c743c404a7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:09.004822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:09.005505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:09.005793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:09.006075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:09.006845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:19:09.007156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:09.008519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:09.011259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:09.017458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 601/1189] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:19:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:09.023531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ae18-6718-48d9-a8e4-dc9fa8fa0019 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:09.039507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:09.122331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634ae18-6718-48d9-a8e4-dc9fa8fa0019 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:09.154080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d17d87-4356-4b59-a077-9df3a3bed3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:09.154941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d17d87-4356-4b59-a077-9df3a3bed3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:09.155225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d17d87-4356-4b59-a077-9df3a3bed3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:09.155508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d17d87-4356-4b59-a077-9df3a3bed3e1 None admin] PATH_INFO: `/v3/users/55f918327e3d4d87809b144c2f30605c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:09.156680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d17d87-4356-4b59-a077-9df3a3bed3e1 None admin] RBAC: Authorizing `identity:get_user(user_id=55f918327e3d4d87809b144c2f30605c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:09.158152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d17d87-4356-4b59-a077-9df3a3bed3e1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:09.159766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 589/1190] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:19:09 2022] GET /identity/v3/users/55f918327e3d4d87809b144c2f30605c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:09.165500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:09.261793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0635cd7-c8a4-457b-9ed4-f9cf8a5d4b38 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:09.293999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84290d83-fc93-4aad-883d-538abcff1d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:09.294840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84290d83-fc93-4aad-883d-538abcff1d0e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:09.295119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84290d83-fc93-4aad-883d-538abcff1d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:09.295392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84290d83-fc93-4aad-883d-538abcff1d0e None admin] PATH_INFO: `/v3/projects/63a4ab5043b94a3c9a5205789cb7497b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:09.296548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84290d83-fc93-4aad-883d-538abcff1d0e None admin] RBAC: Authorizing `identity:get_project(project_id=63a4ab5043b94a3c9a5205789cb7497b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:09.298000 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84290d83-fc93-4aad-883d-538abcff1d0e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:09.299679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 602/1191] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:19:09 2022] GET /identity/v3/projects/63a4ab5043b94a3c9a5205789cb7497b => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:09.305483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d17d87-4356-4b59-a077-9df3a3bed3e1 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:09.387195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:09.404948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d17d87-4356-4b59-a077-9df3a3bed3e1 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:09.437965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d1d6d-c83c-452e-a762-00168706d4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:09.438831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1d6d-c83c-452e-a762-00168706d4f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:09.439110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1d6d-c83c-452e-a762-00168706d4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:09.439379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d1d6d-c83c-452e-a762-00168706d4f1 None admin] PATH_INFO: `/v3/projects/63a4ab5043b94a3c9a5205789cb7497b/users/55f918327e3d4d87809b144c2f30605c/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:09.441303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d1d6d-c83c-452e-a762-00168706d4f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=63a4ab5043b94a3c9a5205789cb7497b, user_id=55f918327e3d4d87809b144c2f30605c, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:09.443075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d1d6d-c83c-452e-a762-00168706d4f1 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:09.526029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 590/1192] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:19:09 2022] PUT /identity/v3/projects/63a4ab5043b94a3c9a5205789cb7497b/users/55f918327e3d4d87809b144c2f30605c/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:19:11.677453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170315fc-c467-471e-b541-6139d0b2442c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:11.677978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170315fc-c467-471e-b541-6139d0b2442c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:11.678071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170315fc-c467-471e-b541-6139d0b2442c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:11.679349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 603/1193] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:19:11.684990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538468f1-a8a9-4584-87e9-7c4636a28c62 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:11.685417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538468f1-a8a9-4584-87e9-7c4636a28c62 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:11.685590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538468f1-a8a9-4584-87e9-7c4636a28c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:11.702325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-538468f1-a8a9-4584-87e9-7c4636a28c62 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:19:11.703506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-538468f1-a8a9-4584-87e9-7c4636a28c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:19:11.766637 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 591/1194] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:19:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:19:11.789023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170315fc-c467-471e-b541-6139d0b2442c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:11.854137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:11.889071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170315fc-c467-471e-b541-6139d0b2442c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:11.921531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbf5f0-9512-4c99-8aab-302b80d2afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:11.922311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbf5f0-9512-4c99-8aab-302b80d2afd5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:11.922598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbf5f0-9512-4c99-8aab-302b80d2afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:11.922874 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbf5f0-9512-4c99-8aab-302b80d2afd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:11.933278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbf5f0-9512-4c99-8aab-302b80d2afd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-39bbf5f0-9512-4c99-8aab-302b80d2afd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:19:11.940814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:19:11.941954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:19:11.942884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 604/1195] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:19:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:19:11.948604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538468f1-a8a9-4584-87e9-7c4636a28c62 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:12.016002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:12.045354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538468f1-a8a9-4584-87e9-7c4636a28c62 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:12.078405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:12.079057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:12.079336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:12.079607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:12.080335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:19:12.080638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:12.082027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:12.089732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 592/1196] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:19:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:12.095513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbf5f0-9512-4c99-8aab-302b80d2afd5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:12.192506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbf5f0-9512-4c99-8aab-302b80d2afd5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:12.216126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:12.224552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de542e9-aba2-444f-acb3-fb0ee0143cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:12.227879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de542e9-aba2-444f-acb3-fb0ee0143cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:12.227879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de542e9-aba2-444f-acb3-fb0ee0143cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:12.227879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de542e9-aba2-444f-acb3-fb0ee0143cd9 None admin] PATH_INFO: `/v3/users/55f918327e3d4d87809b144c2f30605c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:12.227879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de542e9-aba2-444f-acb3-fb0ee0143cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=55f918327e3d4d87809b144c2f30605c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:12.228533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de542e9-aba2-444f-acb3-fb0ee0143cd9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:12.230705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 605/1197] 10.222.0.41 () {56 vars in 1177 bytes} [Sat May 28 03:19:12 2022] GET /identity/v3/users/55f918327e3d4d87809b144c2f30605c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:12.236255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:12.332377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e00e00-35bb-4f2d-8ae9-3216c2273dac None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:12.364551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9911-dbe3-40ef-8fce-439c9f2cabb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:12.365221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9911-dbe3-40ef-8fce-439c9f2cabb0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:12.365504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9911-dbe3-40ef-8fce-439c9f2cabb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:12.365779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9911-dbe3-40ef-8fce-439c9f2cabb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:12.368772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:12.431116 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a9911-dbe3-40ef-8fce-439c9f2cabb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:12.432582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a9911-dbe3-40ef-8fce-439c9f2cabb0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:12.527279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 593/1198] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:19:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:19:12.533227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de542e9-aba2-444f-acb3-fb0ee0143cd9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:12.630790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de542e9-aba2-444f-acb3-fb0ee0143cd9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:12.662809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c4b8b-c500-4883-a5bd-0dfca0b8b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:12.663653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4b8b-c500-4883-a5bd-0dfca0b8b1fa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:12.663935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4b8b-c500-4883-a5bd-0dfca0b8b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:12.664210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c4b8b-c500-4883-a5bd-0dfca0b8b1fa None admin] PATH_INFO: `/v3/projects/63a4ab5043b94a3c9a5205789cb7497b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:12.665394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c4b8b-c500-4883-a5bd-0dfca0b8b1fa None admin] RBAC: Authorizing `identity:get_project(project_id=63a4ab5043b94a3c9a5205789cb7497b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:12.666810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c4b8b-c500-4883-a5bd-0dfca0b8b1fa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:12.668473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 606/1199] 10.222.0.41 () {56 vars in 1186 bytes} [Sat May 28 03:19:12 2022] GET /identity/v3/projects/63a4ab5043b94a3c9a5205789cb7497b => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:12.674519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9911-dbe3-40ef-8fce-439c9f2cabb0 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:19:12.706873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:12.770388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9911-dbe3-40ef-8fce-439c9f2cabb0 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:19:12.802491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bebc3-fbc9-4f17-8b65-66f306580e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:19:12.803065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bebc3-fbc9-4f17-8b65-66f306580e1f None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:19:12.803343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bebc3-fbc9-4f17-8b65-66f306580e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:19:12.803616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bebc3-fbc9-4f17-8b65-66f306580e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:19:12.804428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bebc3-fbc9-4f17-8b65-66f306580e1f None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=63a4ab5043b94a3c9a5205789cb7497b, user.id=55f918327e3d4d87809b144c2f30605c) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:19:12.804734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bebc3-fbc9-4f17-8b65-66f306580e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:19:12.806202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bebc3-fbc9-4f17-8b65-66f306580e1f None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:19:12.812324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 594/1200] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:19:12 2022] GET /identity/v3/role_assignments?user.id=55f918327e3d4d87809b144c2f30605c&scope.project.id=63a4ab5043b94a3c9a5205789cb7497b&role.id=bb0756f8af694633b9743a958eb46e7a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:19:22.658928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=69 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:19:22.809770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=51 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:29.453610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bad21-935f-4f77-8be7-df95c6e9cf3f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:29.454183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bad21-935f-4f77-8be7-df95c6e9cf3f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:29.454183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bad21-935f-4f77-8be7-df95c6e9cf3f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:29.455513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 607/1201] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:20:29.460749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9bc2-0ccb-4953-aed3-743c90490f98 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:29.461229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9bc2-0ccb-4953-aed3-743c90490f98 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:29.461341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9bc2-0ccb-4953-aed3-743c90490f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:29.478598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-213d9bc2-0ccb-4953-aed3-743c90490f98 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:20:29.479778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-213d9bc2-0ccb-4953-aed3-743c90490f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:20:29.544902 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 595/1202] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:20:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:20:29.569574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bad21-935f-4f77-8be7-df95c6e9cf3f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:29.671116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bad21-935f-4f77-8be7-df95c6e9cf3f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:29.706258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:29.707051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:29.707410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:29.707746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:29.722492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:29.723999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:29.741338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:29.742455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:29.743668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:29.743668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:29.743668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:29.743668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:29.743668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 608/1203] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:29.748589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9bc2-0ccb-4953-aed3-743c90490f98 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:29.846339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9bc2-0ccb-4953-aed3-743c90490f98 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:29.873441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:29.879488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e5e32-8aa8-453f-805e-d8096e504588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:29.880136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e5e32-8aa8-453f-805e-d8096e504588 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:29.880418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e5e32-8aa8-453f-805e-d8096e504588 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:29.880763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e5e32-8aa8-453f-805e-d8096e504588 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:29.881452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e5e32-8aa8-453f-805e-d8096e504588 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:29.881759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e5e32-8aa8-453f-805e-d8096e504588 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:29.883158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e5e32-8aa8-453f-805e-d8096e504588 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:29.894425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 596/1204] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:29.900168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:29.953003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:29.998797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bef54-2a06-44c4-9183-e345227b78a5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:30.031997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25252a55-52b0-463c-a4ae-3a0dd35cafc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:30.032656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25252a55-52b0-463c-a4ae-3a0dd35cafc9 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:30.032937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25252a55-52b0-463c-a4ae-3a0dd35cafc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:30.033249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25252a55-52b0-463c-a4ae-3a0dd35cafc9 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:30.033936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25252a55-52b0-463c-a4ae-3a0dd35cafc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:30.035364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25252a55-52b0-463c-a4ae-3a0dd35cafc9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:30.242217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 609/1205] 10.222.0.41 () {60 vars in 1132 bytes} [Sat May 28 03:20:29 2022] POST /identity/v3/users => generated 251 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:20:32.433551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518561f9-5c73-497e-a8dc-66f3a4323682 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:32.434127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518561f9-5c73-497e-a8dc-66f3a4323682 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:32.434127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518561f9-5c73-497e-a8dc-66f3a4323682 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:32.435448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 597/1206] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:20:32.440571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df494a3b-3426-442f-9916-ea06a70d3d0f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:32.440987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df494a3b-3426-442f-9916-ea06a70d3d0f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:32.441158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df494a3b-3426-442f-9916-ea06a70d3d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:32.458125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-df494a3b-3426-442f-9916-ea06a70d3d0f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:20:32.461079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-df494a3b-3426-442f-9916-ea06a70d3d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:20:32.525329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 610/1207] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:20:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:20:32.548151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518561f9-5c73-497e-a8dc-66f3a4323682 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:32.649204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518561f9-5c73-497e-a8dc-66f3a4323682 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:32.682890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55028b56-fbeb-445a-99b4-7a1301cb972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:32.683720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55028b56-fbeb-445a-99b4-7a1301cb972d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:32.683956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55028b56-fbeb-445a-99b4-7a1301cb972d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:32.684230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55028b56-fbeb-445a-99b4-7a1301cb972d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:32.694208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55028b56-fbeb-445a-99b4-7a1301cb972d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-55028b56-fbeb-445a-99b4-7a1301cb972d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:20:32.702063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:32.703193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:32.704401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 598/1208] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:20:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:32.709207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df494a3b-3426-442f-9916-ea06a70d3d0f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:32.748995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:32.805915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df494a3b-3426-442f-9916-ea06a70d3d0f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:32.839224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:32.839886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:32.840212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:32.840494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:32.841265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:32.841575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:32.843826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:32.852662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 611/1209] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:20:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:32.858303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55028b56-fbeb-445a-99b4-7a1301cb972d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:32.908836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:32.956903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55028b56-fbeb-445a-99b4-7a1301cb972d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:32.989118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:32.989957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:32.990234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:32.990504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:33.001057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:33.002527 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:33.019709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:33.020789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:33.021953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:33.021953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:33.021953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:33.021953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:33.021953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 599/1210] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:33.026597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:33.127215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:33.130125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804ea17-4659-4bef-9024-8e8d90a0ae8b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:33.164682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:33.165367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:33.165652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:33.165928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:33.166653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:33.166959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:33.168313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:33.180367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 612/1211] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:33.186011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:33.285651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c887c3-8181-4724-9ede-3f0e3108fdb3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:33.287187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:33.318811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:33.319596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:33.319875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:33.320143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:33.326856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:33.328283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:20:33.342446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:33.346142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:33.347353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 600/1212] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:33.350103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:33.446766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059c967-18a1-498b-ae3c-1bf08f0d0ff8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:33.480332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:33.480987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:33.481309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:33.481594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:33.482422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:33.482759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:33.484840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:33.501725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 613/1213] 10.222.0.41 () {56 vars in 1143 bytes} [Sat May 28 03:20:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:33.507385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:33.610889 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc7e1-642d-467d-8d26-f198ec16a234 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:33.657574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17035a-e9d4-456b-aed8-a4095f7cbbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:33.658555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17035a-e9d4-456b-aed8-a4095f7cbbaa None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:33.659024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17035a-e9d4-456b-aed8-a4095f7cbbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:33.659371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17035a-e9d4-456b-aed8-a4095f7cbbaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:33.696493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:33.734140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17035a-e9d4-456b-aed8-a4095f7cbbaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:33.735593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17035a-e9d4-456b-aed8-a4095f7cbbaa None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:33.831871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 601/1214] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:20:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:20:33.837826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:33.856680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:33.935255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0c897-6398-4bad-99a9-09d85b343bd6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:33.975054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:33.976215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:33.976641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:33.977059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:33.992004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:33.993522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:34.013422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:34.014520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:34.015823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:34.015823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:34.015823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:34.015823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:34.015823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 614/1215] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:34.020423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17035a-e9d4-456b-aed8-a4095f7cbbaa None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:34.080488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:34.118340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17035a-e9d4-456b-aed8-a4095f7cbbaa None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:34.150117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:34.150767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:34.151050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:34.151324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:34.152034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:34.152336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:34.153747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:34.166073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 602/1216] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:34.172406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:34.241515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:34.270470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361dc29-41c6-4e14-bfa3-1892fd22ffd3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:34.304691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:34.305504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:34.310180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:34.310180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:34.313528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:34.314261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:20:34.328012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:34.329092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:34.330087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 615/1217] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:20:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:34.335122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:34.433600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680a272-121f-402a-9bdc-c3b882319118 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:34.455506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:34.466491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbce1d4-7a3c-4024-b0e3-a22f65682a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:34.467151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbce1d4-7a3c-4024-b0e3-a22f65682a4e None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:34.467425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbce1d4-7a3c-4024-b0e3-a22f65682a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:34.467698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbce1d4-7a3c-4024-b0e3-a22f65682a4e None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:34.468486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbce1d4-7a3c-4024-b0e3-a22f65682a4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:34.468794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbce1d4-7a3c-4024-b0e3-a22f65682a4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:34.470261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbce1d4-7a3c-4024-b0e3-a22f65682a4e None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:34.481501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 603/1218] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:20:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:34.488087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:34.585426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef64323-90fc-46e0-8a57-22518fd8a2e5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:34.613059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:34.618736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f6add-84df-44a1-bac8-6caa91084228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:34.619401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6add-84df-44a1-bac8-6caa91084228 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:34.619684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6add-84df-44a1-bac8-6caa91084228 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:34.619958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6add-84df-44a1-bac8-6caa91084228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:34.620776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f6add-84df-44a1-bac8-6caa91084228 None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=1e771dc83fa34d79947647e56ca338a6) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:34.621086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f6add-84df-44a1-bac8-6caa91084228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:34.622587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f6add-84df-44a1-bac8-6caa91084228 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:34.628473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 616/1219] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:20:34 2022] GET /identity/v3/role_assignments?user.id=1e771dc83fa34d79947647e56ca338a6&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:36.803530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8854654-639f-4dbd-b8a4-c52f6922018e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:36.804107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8854654-639f-4dbd-b8a4-c52f6922018e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:36.804107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8854654-639f-4dbd-b8a4-c52f6922018e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:36.805475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 604/1220] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:20:36.810720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3931-f292-422f-b708-43513cd19e4f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:36.811354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3931-f292-422f-b708-43513cd19e4f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:36.811354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e3931-f292-422f-b708-43513cd19e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:36.828190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1b1e3931-f292-422f-b708-43513cd19e4f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:20:36.829444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b1e3931-f292-422f-b708-43513cd19e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:20:36.895247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 617/1221] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:20:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:20:36.918496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8854654-639f-4dbd-b8a4-c52f6922018e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:37.020627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8854654-639f-4dbd-b8a4-c52f6922018e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:37.053085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcc1ee-ce16-4447-873c-3670013a251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:37.053952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc1ee-ce16-4447-873c-3670013a251d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:37.054176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc1ee-ce16-4447-873c-3670013a251d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:37.054446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcc1ee-ce16-4447-873c-3670013a251d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:37.064722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcc1ee-ce16-4447-873c-3670013a251d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-e1dcc1ee-ce16-4447-873c-3670013a251d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:20:37.072392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:37.073559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:37.074682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 605/1222] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:20:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:37.080440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e3931-f292-422f-b708-43513cd19e4f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:37.178696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e3931-f292-422f-b708-43513cd19e4f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:37.201835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:37.212817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:37.213506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:37.213792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:37.214072 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:37.214817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:37.215127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:37.216483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:37.224589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 618/1223] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:20:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:37.230425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcc1ee-ce16-4447-873c-3670013a251d None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:37.274400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:37.329600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcc1ee-ce16-4447-873c-3670013a251d None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:37.362092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:37.362866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:37.363145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:37.363416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:37.373773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:37.375741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:37.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:37.394159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:37.395137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:37.395137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:37.395137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:37.395137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:37.395137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 606/1224] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:37.399941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:37.497256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1f943-1cc7-4cfa-b7d3-50790e92de7d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:37.530066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:37.530734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:37.531018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:37.531295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:37.532117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:37.532418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:37.533802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:37.545112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 619/1225] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:37.555040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:37.642592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:37.648964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683175a7-d99e-46e9-b325-a48b15f36a5a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:37.684266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:37.685084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:37.685449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:37.685768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] PATH_INFO: `/v3/users/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:37.693571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:37.695007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:20:37.709523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:37.710638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:37.711793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 607/1226] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:37.716592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:37.768001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:37.818483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef36be-1462-4f70-9f74-2d0bce90621f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:37.851268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:37.851932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:37.852210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:37.852479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:37.853324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:37.853647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:37.855050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:37.874158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 620/1227] 10.222.0.41 () {56 vars in 1143 bytes} [Sat May 28 03:20:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:37.880197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:37.981810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cb889-9da3-42cb-92ea-f72595fc7aec None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:38.014922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:38.015705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:38.015989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:38.016263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:38.026058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:38.027510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:38.044980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:38.046126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:38.047575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:38.047575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:38.047575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:38.047575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:38.047575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 608/1228] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:38.052540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:38.150383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5631d-1fda-4f41-af55-79c57352283f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:38.152674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:38.187974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:38.188552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:38.188835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:38.189142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:38.189855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:38.190162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:38.191522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:38.203061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 621/1229] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:38.208631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:38.217319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:38.306190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec08465-334a-4699-bf9e-8f8d49c8f4eb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:38.339872 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:38.340656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:38.340934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:38.341236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:38.348228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:38.350029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:20:38.363767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:38.364825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:38.366043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 609/1230] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:20:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:38.370831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:38.467645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4fb66-03f8-4d32-9350-3d8ef7d8667b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:38.500940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca06d75-a5d5-4fe9-97b8-947e88584a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:38.501617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca06d75-a5d5-4fe9-97b8-947e88584a64 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:38.501907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca06d75-a5d5-4fe9-97b8-947e88584a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:38.502176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca06d75-a5d5-4fe9-97b8-947e88584a64 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:38.502950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca06d75-a5d5-4fe9-97b8-947e88584a64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:38.503256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca06d75-a5d5-4fe9-97b8-947e88584a64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:38.504664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca06d75-a5d5-4fe9-97b8-947e88584a64 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:38.516779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 622/1231] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:20:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:38.522618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:38.585630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:38.618981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf9849-1173-45ff-bde0-e48f0784ac7c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:38.652177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d2085-3b3e-402a-872c-f948a22e5f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:38.653047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d2085-3b3e-402a-872c-f948a22e5f28 None admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:38.653366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d2085-3b3e-402a-872c-f948a22e5f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:38.653638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d2085-3b3e-402a-872c-f948a22e5f28 None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/1e771dc83fa34d79947647e56ca338a6/roles/e648d64a4d3d440ba9feaa50a398c2e6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:38.667720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d2085-3b3e-402a-872c-f948a22e5f28 None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=1e771dc83fa34d79947647e56ca338a6, role_id=e648d64a4d3d440ba9feaa50a398c2e6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:38.669194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d2085-3b3e-402a-872c-f948a22e5f28 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:38.744317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 610/1232] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:20:38 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/1e771dc83fa34d79947647e56ca338a6/roles/e648d64a4d3d440ba9feaa50a398c2e6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:20:40.884792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a506c-cc12-4ea9-9d32-06ac5954709a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:40.885404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a506c-cc12-4ea9-9d32-06ac5954709a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:40.885404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a506c-cc12-4ea9-9d32-06ac5954709a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:40.886731 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 623/1233] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:20:40.892635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f39bdd-8229-493d-8955-4eff2de5f676 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:40.893040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f39bdd-8229-493d-8955-4eff2de5f676 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:40.893908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f39bdd-8229-493d-8955-4eff2de5f676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:40.910024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-08f39bdd-8229-493d-8955-4eff2de5f676 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:20:40.911346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-08f39bdd-8229-493d-8955-4eff2de5f676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:20:40.987339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 611/1234] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:20:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:20:41.012443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a506c-cc12-4ea9-9d32-06ac5954709a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:41.038334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:41.113991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a506c-cc12-4ea9-9d32-06ac5954709a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:41.147935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e918203-ccf1-4f2a-96de-b4ecd327efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:41.148657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e918203-ccf1-4f2a-96de-b4ecd327efbf None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:41.148939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e918203-ccf1-4f2a-96de-b4ecd327efbf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:41.149242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e918203-ccf1-4f2a-96de-b4ecd327efbf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:41.159307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e918203-ccf1-4f2a-96de-b4ecd327efbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3e918203-ccf1-4f2a-96de-b4ecd327efbf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:20:41.167128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:41.172488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:41.173558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 624/1235] 10.222.0.41 () {56 vars in 1102 bytes} [Sat May 28 03:20:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:41.175601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f39bdd-8229-493d-8955-4eff2de5f676 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:41.220664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:41.273530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f39bdd-8229-493d-8955-4eff2de5f676 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:41.306308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:41.306974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:41.307278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:41.307531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:41.308280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] RBAC: Adding query filter params (name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:41.308588 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:41.310697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:41.321256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 612/1236] 10.222.0.41 () {56 vars in 1103 bytes} [Sat May 28 03:20:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:41.327654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e918203-ccf1-4f2a-96de-b4ecd327efbf None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:41.411882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:41.427524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e918203-ccf1-4f2a-96de-b4ecd327efbf None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:41.460020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:41.460803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:41.461087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:41.461395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:41.471707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:41.474689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:41.490589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:41.491585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:41.492567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:41.492567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:41.492567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:41.492567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:41.492567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 625/1237] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:41.497512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:41.595180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fa62e-8cc4-4db1-9764-55a75480013c None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:41.603958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:41.634112 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:41.634808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:41.635091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:41.635363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:41.636075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:41.636382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:41.637788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:41.649403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 613/1238] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:41.655105 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:41.756825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2726e-144d-4bcf-a97a-53d6ba674ba9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:41.790377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:41.791168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:41.791450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:41.791724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:41.794512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:41.800338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:41.801872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:20:41.816029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:41.817157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:41.818307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 626/1239] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:41.823061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:41.926057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19423fe-ddf3-43e0-b772-e111b000d519 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:41.963372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:41.964069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:41.964355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:41.964667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:41.965554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:41.965880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:41.967313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:41.985872 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 614/1240] 10.222.0.41 () {56 vars in 1143 bytes} [Sat May 28 03:20:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:41.991607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:42.091016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c5878-732a-41b7-bed5-2404cd79f579 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:42.124448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70047da-c869-405b-b206-1c1847006eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:42.125193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70047da-c869-405b-b206-1c1847006eea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:42.125525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70047da-c869-405b-b206-1c1847006eea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:42.125806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70047da-c869-405b-b206-1c1847006eea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:42.186848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:42.193506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70047da-c869-405b-b206-1c1847006eea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:42.194962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70047da-c869-405b-b206-1c1847006eea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:42.291239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 627/1241] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:20:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:20:42.297090 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:42.324065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:42.393229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ebbab-afb8-49f9-a379-0a31e2840b25 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:42.426204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:42.426982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:42.427267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:42.427542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:42.437181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:42.440226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:42.455870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:42.456748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:42.463303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:42.463303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:42.463303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:42.463303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:42.463303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 615/1242] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:42.463806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70047da-c869-405b-b206-1c1847006eea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:42.555398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:42.560987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70047da-c869-405b-b206-1c1847006eea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:42.595108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:42.595788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:42.596089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:42.596388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:42.597198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:42.597508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:42.598940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:42.611272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 628/1243] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:42.616978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:42.692394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:42.713044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbc775-18aa-452e-80a4-a5ef0943c581 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:42.746826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:42.747610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:42.747892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:42.748164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:42.754812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:42.757616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:20:42.770500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:42.771543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:42.772665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 616/1244] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:20:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:42.777462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:42.877243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d092d-5138-43fb-afc7-18f14f20d4b1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:42.911079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2286a6-3e16-44f3-9484-4fb21b307661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:42.911745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2286a6-3e16-44f3-9484-4fb21b307661 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:42.912032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2286a6-3e16-44f3-9484-4fb21b307661 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:42.912355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2286a6-3e16-44f3-9484-4fb21b307661 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:42.913159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2286a6-3e16-44f3-9484-4fb21b307661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:42.913477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2286a6-3e16-44f3-9484-4fb21b307661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:42.914942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2286a6-3e16-44f3-9484-4fb21b307661 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:42.927729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 629/1245] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:20:42.933964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:43.035080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6a914-ca99-43dc-9f32-ad78477d6d08 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:43.068428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49e4f3-931a-46e9-a31f-17f70eb9851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:43.069169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49e4f3-931a-46e9-a31f-17f70eb9851d None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:43.069459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49e4f3-931a-46e9-a31f-17f70eb9851d None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:43.069736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49e4f3-931a-46e9-a31f-17f70eb9851d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:43.070556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49e4f3-931a-46e9-a31f-17f70eb9851d None admin] RBAC: Adding query filter params (role.id=e648d64a4d3d440ba9feaa50a398c2e6, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=1e771dc83fa34d79947647e56ca338a6) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:43.070869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49e4f3-931a-46e9-a31f-17f70eb9851d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:43.072312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49e4f3-931a-46e9-a31f-17f70eb9851d None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:43.078649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 617/1246] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:20:42 2022] GET /identity/v3/role_assignments?user.id=1e771dc83fa34d79947647e56ca338a6&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=e648d64a4d3d440ba9feaa50a398c2e6 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:45.304088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a748c2-725f-424c-aab3-b73e5067ad58 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:45.304639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a748c2-725f-424c-aab3-b73e5067ad58 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:45.304639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a748c2-725f-424c-aab3-b73e5067ad58 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:45.306011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 630/1247] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:20:45.311226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec4fd-c9e7-4c6b-96b8-ac9e432169b6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:45.311604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec4fd-c9e7-4c6b-96b8-ac9e432169b6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:45.312342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec4fd-c9e7-4c6b-96b8-ac9e432169b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:45.317337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:45.329745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0fdec4fd-c9e7-4c6b-96b8-ac9e432169b6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:20:45.330910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0fdec4fd-c9e7-4c6b-96b8-ac9e432169b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:20:45.396765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 618/1248] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:20:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:20:45.420148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a748c2-725f-424c-aab3-b73e5067ad58 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:45.438376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:45.522002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a748c2-725f-424c-aab3-b73e5067ad58 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:45.554850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c676e2-ce31-4eba-811a-6d2164363ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:45.555568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c676e2-ce31-4eba-811a-6d2164363ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:45.555855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c676e2-ce31-4eba-811a-6d2164363ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:45.556154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c676e2-ce31-4eba-811a-6d2164363ba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:45.566830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c676e2-ce31-4eba-811a-6d2164363ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-00c676e2-ce31-4eba-811a-6d2164363ba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:20:45.574559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:45.575640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:45.576725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 631/1249] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:20:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:45.582401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec4fd-c9e7-4c6b-96b8-ac9e432169b6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:45.689514 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec4fd-c9e7-4c6b-96b8-ac9e432169b6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:45.718645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:45.723377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:45.724037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:45.724322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:45.724863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:45.725629 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:45.731009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:45.731009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:45.735553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 619/1250] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:20:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:45.741271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c676e2-ce31-4eba-811a-6d2164363ba9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:45.820393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:45.842149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c676e2-ce31-4eba-811a-6d2164363ba9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:45.875256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:45.876035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:45.876317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:45.876589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:45.887458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:45.888896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:45.905648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:45.906551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:45.908313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:45.908313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:45.908313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:45.908313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:45.908313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 632/1251] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:45.912934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:46.010652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfcffc-228c-41c3-882b-cc3f97b93dfb None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:46.042866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:46.043506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:46.043783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:46.044053 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:46.044754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:46.045057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:46.047054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:46.058371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 620/1252] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:46.064006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:46.161860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ba358-6ca9-4fb7-a4e4-dad507d7b89a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:46.186982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:46.197036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:46.197860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:46.198139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:46.198431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:46.206430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:46.207853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:20:46.222419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:46.226058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:46.227209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 633/1253] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:46.229636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:46.284202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:46.326945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525b269-f41a-42e1-abd5-eb9d92bf6e57 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:46.361121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:46.362439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:46.362439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:46.362834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:46.364038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:46.364431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:46.366000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:46.384849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 621/1254] 10.222.0.41 () {56 vars in 1143 bytes} [Sat May 28 03:20:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:46.391060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:46.489855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b58cf9-1c2e-470a-b355-68c7f5588524 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:46.522241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90ad05-05ed-4ecb-95ca-62f81a198426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:46.522869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ad05-05ed-4ecb-95ca-62f81a198426 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:46.523152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ad05-05ed-4ecb-95ca-62f81a198426 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:46.523425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90ad05-05ed-4ecb-95ca-62f81a198426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:46.576260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:46.588547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90ad05-05ed-4ecb-95ca-62f81a198426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:46.590064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90ad05-05ed-4ecb-95ca-62f81a198426 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:46.686604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 634/1255] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:20:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:20:46.692526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:46.788792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84105379-a204-499e-a79a-d089295dbf22 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:46.802281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:46.822585 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:46.823362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:46.823644 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:46.823917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:46.834254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:46.835696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:46.852827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:46.856225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:46.857485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:46.857485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:46.857485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:46.857485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:46.857485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 622/1256] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:46.859583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90ad05-05ed-4ecb-95ca-62f81a198426 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:46.945932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:46.958142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90ad05-05ed-4ecb-95ca-62f81a198426 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:46.991205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:46.991864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:46.992147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:46.992448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:46.993273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:46.993563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:46.994920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:47.006430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 635/1257] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:47.012211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:47.111243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0bcf3-082f-4818-8a1f-76666d173eb2 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:47.145209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:47.145982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:47.146262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:47.146532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:47.152985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:47.154514 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:20:47.167971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:47.169014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:47.169983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 623/1258] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:20:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:47.175541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:47.275836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757ddb0-618c-4b19-9eaf-135330942093 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:47.308350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71c5ba-4fb1-487f-a5a3-9cd6f18d429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:47.308999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71c5ba-4fb1-487f-a5a3-9cd6f18d429e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:47.309322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71c5ba-4fb1-487f-a5a3-9cd6f18d429e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:47.309598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71c5ba-4fb1-487f-a5a3-9cd6f18d429e None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:47.310372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71c5ba-4fb1-487f-a5a3-9cd6f18d429e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:47.310681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71c5ba-4fb1-487f-a5a3-9cd6f18d429e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:47.312700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71c5ba-4fb1-487f-a5a3-9cd6f18d429e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:47.324729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 636/1259] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:47.331579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:47.356258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:47.427848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58d180-b7dd-4952-b118-0e19f61547f5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:47.460551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1abbd9-7b02-4bb5-8f21-56ca2f627882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:47.461216 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1abbd9-7b02-4bb5-8f21-56ca2f627882 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:47.461500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1abbd9-7b02-4bb5-8f21-56ca2f627882 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:47.461775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1abbd9-7b02-4bb5-8f21-56ca2f627882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:47.462587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1abbd9-7b02-4bb5-8f21-56ca2f627882 None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=1e771dc83fa34d79947647e56ca338a6) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:47.462899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1abbd9-7b02-4bb5-8f21-56ca2f627882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:47.464411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1abbd9-7b02-4bb5-8f21-56ca2f627882 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:47.469750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 624/1260] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:20:47 2022] GET /identity/v3/role_assignments?user.id=1e771dc83fa34d79947647e56ca338a6&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=bb0756f8af694633b9743a958eb46e7a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:49.630450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402a2d9-5ba3-49f4-8757-0e59910e6080 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:49.630984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402a2d9-5ba3-49f4-8757-0e59910e6080 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:49.630984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402a2d9-5ba3-49f4-8757-0e59910e6080 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:49.632422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 637/1261] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:20:49.637724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6b47e-a44f-4d86-8484-28f9b3ea1e10 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:49.638105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6b47e-a44f-4d86-8484-28f9b3ea1e10 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:49.638284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6b47e-a44f-4d86-8484-28f9b3ea1e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:49.655046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a3f6b47e-a44f-4d86-8484-28f9b3ea1e10 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:20:49.656771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a3f6b47e-a44f-4d86-8484-28f9b3ea1e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:20:49.722074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 625/1262] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:20:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:20:49.744700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402a2d9-5ba3-49f4-8757-0e59910e6080 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:49.756351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:49.851328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402a2d9-5ba3-49f4-8757-0e59910e6080 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:49.885130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6633f-cdb0-4093-abac-985b67dce33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:49.885923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6633f-cdb0-4093-abac-985b67dce33e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:49.886207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6633f-cdb0-4093-abac-985b67dce33e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:49.886481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6633f-cdb0-4093-abac-985b67dce33e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:49.897286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e6633f-cdb0-4093-abac-985b67dce33e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-61e6633f-cdb0-4093-abac-985b67dce33e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:20:49.905163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:49.906082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:49.907432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 638/1263] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:20:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:49.912411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6b47e-a44f-4d86-8484-28f9b3ea1e10 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:49.930665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:50.012481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6b47e-a44f-4d86-8484-28f9b3ea1e10 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:50.047006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:50.047676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:50.047961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:50.048234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:50.048977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:50.049325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:50.050700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:50.058990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 626/1264] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:20:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:50.065446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6633f-cdb0-4093-abac-985b67dce33e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:50.131403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:50.163573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6633f-cdb0-4093-abac-985b67dce33e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:50.196131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:50.196913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:50.197300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:50.197585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:50.211298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:50.212781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:50.230030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:50.231089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:50.232107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:50.232107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:50.232107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:50.232107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:50.232107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 639/1265] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:50.237506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:50.326375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:50.346564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9cb62-83d4-4368-a1d9-a4b3476e351b None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:50.379619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:50.380277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:50.380560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:50.380836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:50.381583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:50.381889 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:50.383386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:50.394734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 627/1266] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:50.400498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:50.500867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0195747-3e06-4f5c-bdeb-e2000a8b7294 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:50.520410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:50.534501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:50.535296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:50.535573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:50.535844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:50.543291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:50.544961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:20:50.559108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:50.560207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:50.561191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 640/1267] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:50.566245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:50.663487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25772ca-b51f-4450-bb22-0294e2087308 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:50.698910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:50.699571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:50.699954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:50.700233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:50.701065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:50.701406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:50.702838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:50.711383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:50.720160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 628/1268] 10.222.0.41 () {56 vars in 1143 bytes} [Sat May 28 03:20:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:50.726062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:50.827398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7718a02-3025-4230-b297-c505f52f3a2b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:50.861455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:50.862240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:50.862535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:50.862811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:50.873275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:50.874792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:50.892234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:50.893336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:50.894537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:50.894537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:50.894537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:50.894537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:50.894537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 641/1269] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:50.899220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:50.997202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ec0a6-c568-4655-b210-3b4cf10c4cff None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:51.031127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:51.031811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:51.032094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:51.032367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:51.033070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:51.033403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:51.034814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:51.045771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 629/1270] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:51.051513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:51.083671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:51.151003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ce922-f928-4009-a077-0177e6220656 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:51.184784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:51.185634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:51.185924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:51.186204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:51.192907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:51.194382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:20:51.208999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:51.210093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:51.211074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 642/1271] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:20:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:51.216151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:51.274168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:51.316591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d2066-d7d0-4d9b-b650-66051c34b700 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:51.349165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d67116-cab0-4895-a29b-e7b42940dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:51.351121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d67116-cab0-4895-a29b-e7b42940dc2a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:51.351917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d67116-cab0-4895-a29b-e7b42940dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:51.352229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d67116-cab0-4895-a29b-e7b42940dc2a None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:51.353055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d67116-cab0-4895-a29b-e7b42940dc2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:51.353396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d67116-cab0-4895-a29b-e7b42940dc2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:51.355092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d67116-cab0-4895-a29b-e7b42940dc2a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:51.367575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 630/1272] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:20:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:51.373650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:51.462479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:51.470921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c6ce5-37f0-4e2c-81cc-591281495e61 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:51.505775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509350d-ee7a-4da1-9615-2b75db5e5b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:51.506646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509350d-ee7a-4da1-9615-2b75db5e5b8f None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:51.506922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509350d-ee7a-4da1-9615-2b75db5e5b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:51.507192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509350d-ee7a-4da1-9615-2b75db5e5b8f None admin] PATH_INFO: `/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/1e771dc83fa34d79947647e56ca338a6/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:51.509095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509350d-ee7a-4da1-9615-2b75db5e5b8f None admin] RBAC: Authorizing `identity:create_grant(project_id=eeb46e722ef6478eb40cec80b8278e24, user_id=1e771dc83fa34d79947647e56ca338a6, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:51.510534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509350d-ee7a-4da1-9615-2b75db5e5b8f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:51.601558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 643/1273] 10.222.0.41 () {58 vars in 1439 bytes} [Sat May 28 03:20:51 2022] PUT /identity/v3/projects/eeb46e722ef6478eb40cec80b8278e24/users/1e771dc83fa34d79947647e56ca338a6/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:20:53.797055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048610f-b7c9-4b81-8f28-45c1ba52626a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:53.797629 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048610f-b7c9-4b81-8f28-45c1ba52626a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:53.797725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048610f-b7c9-4b81-8f28-45c1ba52626a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:53.799095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 631/1274] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:20:53.804412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf3aed-2eab-4465-a612-48ac3716896d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:53.804847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf3aed-2eab-4465-a612-48ac3716896d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:53.804962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf3aed-2eab-4465-a612-48ac3716896d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:53.821181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-76cf3aed-2eab-4465-a612-48ac3716896d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:20:53.822350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-76cf3aed-2eab-4465-a612-48ac3716896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:20:53.891308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 644/1275] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:20:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 03:20:53.915330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048610f-b7c9-4b81-8f28-45c1ba52626a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:54.016919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048610f-b7c9-4b81-8f28-45c1ba52626a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:54.034487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:54.050565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2afe73-b6cf-4f8a-9e1b-ddbd0efbeb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:54.051293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afe73-b6cf-4f8a-9e1b-ddbd0efbeb56 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:54.051575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afe73-b6cf-4f8a-9e1b-ddbd0efbeb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:54.051848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2afe73-b6cf-4f8a-9e1b-ddbd0efbeb56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:54.061634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2afe73-b6cf-4f8a-9e1b-ddbd0efbeb56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-4b2afe73-b6cf-4f8a-9e1b-ddbd0efbeb56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 03:20:54.069158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:54.070403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:54.071247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 632/1276] 10.222.0.41 () {56 vars in 1096 bytes} [Sat May 28 03:20:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:54.076408 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf3aed-2eab-4465-a612-48ac3716896d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:54.158355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:54.176113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf3aed-2eab-4465-a612-48ac3716896d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:54.212616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:54.213314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:54.213597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:54.213872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:54.214625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:54.214937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:54.216382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:54.225628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 645/1277] 10.222.0.41 () {56 vars in 1099 bytes} [Sat May 28 03:20:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:54.234028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2afe73-b6cf-4f8a-9e1b-ddbd0efbeb56 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:54.334708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2afe73-b6cf-4f8a-9e1b-ddbd0efbeb56 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:54.366601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:54.367378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:54.367658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:54.367928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:54.378671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:54.380118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:54.397160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:54.398213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:54.399379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:54.399379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:54.399379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:54.399379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:54.399379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 633/1278] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:54.404075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:54.503136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d6ef-8a5f-4eaf-9bb8-aa37d53fccfe None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:54.536268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:54.536920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:54.537265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:54.537545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:54.538257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:54.538984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:54.539908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:54.551402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 646/1279] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:54.557129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:54.590409 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:54.654538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768987d-0d64-40c2-a8ab-152848743744 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:54.688881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:54.689759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:54.690046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:54.690316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:54.697208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:54.698646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 03:20:54.713079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:54.714202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:54.715365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 634/1280] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:54.720248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:54.724080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:54.819560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b66b3-06e6-4a0b-bf99-d1d8d16a01d8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:54.852241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:54.852835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:54.853141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:54.853427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:54.854251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:54.854560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:54.855975 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:54.873334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 647/1281] 10.222.0.41 () {56 vars in 1143 bytes} [Sat May 28 03:20:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:54.879604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:54.969673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:54.979140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465dd899-6786-4172-9444-ffd6fd678d1a None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:55.012413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d83d-e506-4a3e-8df5-480341140f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:55.013045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d83d-e506-4a3e-8df5-480341140f19 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:55.013352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d83d-e506-4a3e-8df5-480341140f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:55.013622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d83d-e506-4a3e-8df5-480341140f19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:55.079295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0d83d-e506-4a3e-8df5-480341140f19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:55.080744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0d83d-e506-4a3e-8df5-480341140f19 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:55.165351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:55.175623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 635/1282] 10.222.0.41 () {58 vars in 1282 bytes} [Sat May 28 03:20:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 03:20:55.181539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:55.242031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:55.278842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae423-d527-4201-9906-48f9dd838472 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:55.314103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:55.314886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:55.315164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:55.315433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:55.328041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:55.329514 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 03:20:55.347225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:55.348264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:55.349444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 03:20:55.349444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 03:20:55.349444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 03:20:55.349444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:55.349444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 648/1283] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:55.354382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d83d-e506-4a3e-8df5-480341140f19 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:55.452898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d83d-e506-4a3e-8df5-480341140f19 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:55.486153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:55.486839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:55.487105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:55.487382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:55.488098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:55.488404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:55.489877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:55.501737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 636/1284] 10.222.0.41 () {56 vars in 1109 bytes} [Sat May 28 03:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:55.507627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:55.622892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f47f1-e0f2-4d81-a59f-cfd175201466 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:55.639647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:55.661160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:55.661977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:55.662253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:55.662697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:55.670745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:55.672425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 03:20:55.687340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:55.688525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application May 28 03:20:55.689486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 649/1285] 10.222.0.41 () {56 vars in 1111 bytes} [Sat May 28 03:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:55.694767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:55.753021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:55.805202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f49e2-3ea1-498a-8968-582b44672ed5 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:55.841388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ac834-4907-4698-90b6-e35dee2fdbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:55.842057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac834-4907-4698-90b6-e35dee2fdbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:55.842343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac834-4907-4698-90b6-e35dee2fdbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:55.842615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ac834-4907-4698-90b6-e35dee2fdbe0 None admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:55.843393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ac834-4907-4698-90b6-e35dee2fdbe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:55.843696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ac834-4907-4698-90b6-e35dee2fdbe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:55.845137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ac834-4907-4698-90b6-e35dee2fdbe0 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:55.858837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 637/1286] 10.222.0.41 () {56 vars in 1148 bytes} [Sat May 28 03:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:55.865568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:55.975306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaac7c-bc7e-4d09-81a7-69bd5dc64921 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:56.011912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fda9ac-b598-433b-9dee-e1b88b31e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:56.011912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda9ac-b598-433b-9dee-e1b88b31e90e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:56.011912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda9ac-b598-433b-9dee-e1b88b31e90e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:56.011912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda9ac-b598-433b-9dee-e1b88b31e90e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:56.012434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fda9ac-b598-433b-9dee-e1b88b31e90e None admin] RBAC: Adding query filter params (role.id=bb0756f8af694633b9743a958eb46e7a, scope.project.id=eeb46e722ef6478eb40cec80b8278e24, user.id=1e771dc83fa34d79947647e56ca338a6) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:56.012434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fda9ac-b598-433b-9dee-e1b88b31e90e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:56.013160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fda9ac-b598-433b-9dee-e1b88b31e90e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:56.019587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 650/1287] 10.222.0.41 () {56 vars in 1374 bytes} [Sat May 28 03:20:55 2022] GET /identity/v3/role_assignments?user.id=1e771dc83fa34d79947647e56ca338a6&scope.project.id=eeb46e722ef6478eb40cec80b8278e24&role.id=bb0756f8af694633b9743a958eb46e7a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:20:58.265728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81383e-1f0e-4da9-9f66-ef80118c1d51 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:58.266435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81383e-1f0e-4da9-9f66-ef80118c1d51 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:58.266435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81383e-1f0e-4da9-9f66-ef80118c1d51 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:58.267865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 638/1288] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:20:58.273231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77356e0d-b086-4ee8-a2eb-4b29166907ba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:58.273770 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77356e0d-b086-4ee8-a2eb-4b29166907ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:58.273919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77356e0d-b086-4ee8-a2eb-4b29166907ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:58.291821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-77356e0d-b086-4ee8-a2eb-4b29166907ba None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:20:58.293017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-77356e0d-b086-4ee8-a2eb-4b29166907ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:20:58.322754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:58.360037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 651/1289] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:20:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:20:58.381713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81383e-1f0e-4da9-9f66-ef80118c1d51 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:58.478455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81383e-1f0e-4da9-9f66-ef80118c1d51 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:58.488516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:20:58.510138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:58.510914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:58.511191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:58.511459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:58.512092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:58.513480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:20:58.525438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:20:58.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:20:58.528056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 639/1290] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:20:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:20:58.532970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77356e0d-b086-4ee8-a2eb-4b29166907ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:58.635547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77356e0d-b086-4ee8-a2eb-4b29166907ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:58.668690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4da06-d5f8-43a2-aa4a-fd6b7ead4405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:58.669375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4da06-d5f8-43a2-aa4a-fd6b7ead4405 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:58.669660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4da06-d5f8-43a2-aa4a-fd6b7ead4405 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:58.669937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4da06-d5f8-43a2-aa4a-fd6b7ead4405 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:58.670636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4da06-d5f8-43a2-aa4a-fd6b7ead4405 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:58.670962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4da06-d5f8-43a2-aa4a-fd6b7ead4405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:58.672322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4da06-d5f8-43a2-aa4a-fd6b7ead4405 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:58.677057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 652/1291] 10.222.0.41 () {56 vars in 1116 bytes} [Sat May 28 03:20:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:20:58.682855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:20:58.779107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d27be-d20e-4efb-921c-9c5269f090b3 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:20:58.812153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f192a-f309-4a63-841c-ceedb01d35ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:20:58.812806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f192a-f309-4a63-841c-ceedb01d35ab None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:20:58.813091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f192a-f309-4a63-841c-ceedb01d35ab None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:20:58.813403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f192a-f309-4a63-841c-ceedb01d35ab None admin] PATH_INFO: `/v3/services` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:20:58.814099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f192a-f309-4a63-841c-ceedb01d35ab None admin] RBAC: Adding query filter params (type=placement) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:20:58.814408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f192a-f309-4a63-841c-ceedb01d35ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:20:58.815777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f192a-f309-4a63-841c-ceedb01d35ab None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:20:58.820335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 640/1292] 10.222.0.41 () {56 vars in 1116 bytes} [Sat May 28 03:20:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:21:00.976407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad44f8a-ab44-4bce-a976-10503698f220 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:00.976943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad44f8a-ab44-4bce-a976-10503698f220 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:00.976943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad44f8a-ab44-4bce-a976-10503698f220 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:00.978341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 653/1293] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:21:00.983451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c76beb-9119-4838-801f-becf37d97aa8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:00.983875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c76beb-9119-4838-801f-becf37d97aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:00.984026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c76beb-9119-4838-801f-becf37d97aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:01.001217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-23c76beb-9119-4838-801f-becf37d97aa8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:21:01.002394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-23c76beb-9119-4838-801f-becf37d97aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:21:01.031230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:21:01.067577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 641/1294] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:21:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:21:01.094569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad44f8a-ab44-4bce-a976-10503698f220 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:21:01.125376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:21:01.192674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad44f8a-ab44-4bce-a976-10503698f220 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:21:01.226025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac32b96-adc0-49d2-ae6e-6c6f6ec66535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:21:01.226724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac32b96-adc0-49d2-ae6e-6c6f6ec66535 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:01.227008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac32b96-adc0-49d2-ae6e-6c6f6ec66535 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:01.227287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac32b96-adc0-49d2-ae6e-6c6f6ec66535 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:01.228005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac32b96-adc0-49d2-ae6e-6c6f6ec66535 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:21:01.229440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac32b96-adc0-49d2-ae6e-6c6f6ec66535 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:21:01.321401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 654/1295] 10.222.0.41 () {60 vars in 1141 bytes} [Sat May 28 03:21:01 2022] POST /identity/v3/services => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:21:03.491472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1e64-d1fd-44b7-a4a5-e2275b0d8ab2 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:03.492071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1e64-d1fd-44b7-a4a5-e2275b0d8ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:03.492071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1e64-d1fd-44b7-a4a5-e2275b0d8ab2 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:03.493417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 642/1296] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:21:03.498735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602045c5-ecc1-4f3f-bb12-d594f3dd4faf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:03.501370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602045c5-ecc1-4f3f-bb12-d594f3dd4faf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:03.501370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602045c5-ecc1-4f3f-bb12-d594f3dd4faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:03.515910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-602045c5-ecc1-4f3f-bb12-d594f3dd4faf None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:21:03.517092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-602045c5-ecc1-4f3f-bb12-d594f3dd4faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:21:03.588420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 655/1297] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:21:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:21:03.612364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c1e64-d1fd-44b7-a4a5-e2275b0d8ab2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:21:03.711036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c1e64-d1fd-44b7-a4a5-e2275b0d8ab2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:21:03.744667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:21:03.745426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:03.745711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:03.745989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:03.746642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:21:03.748026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:21:03.759659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:21:03.760919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:21:03.761999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 643/1298] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:21:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:21:03.767332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602045c5-ecc1-4f3f-bb12-d594f3dd4faf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:21:03.785940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:21:03.864309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602045c5-ecc1-4f3f-bb12-d594f3dd4faf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:21:03.897652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:21:03.898304 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:03.898590 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:03.898870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:03.899564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:21:03.899873 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:21:03.901258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:21:03.909495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 656/1299] 10.222.0.41 () {56 vars in 1116 bytes} [Sat May 28 03:21:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:21:03.915407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:21:03.946917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:21:04.013110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e19d36-6921-4298-b554-544b8bb0b3e7 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:21:04.046545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f390a-3442-4c65-855f-49b7ed9066d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:21:04.047197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f390a-3442-4c65-855f-49b7ed9066d9 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:04.047482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f390a-3442-4c65-855f-49b7ed9066d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:04.047761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f390a-3442-4c65-855f-49b7ed9066d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:04.048543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f390a-3442-4c65-855f-49b7ed9066d9 None admin] RBAC: Adding query filter params (interface=public, service_id=c8d392e1566448b7a65cb37acb52295c, region_id=RegionOne) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:21:04.048857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f390a-3442-4c65-855f-49b7ed9066d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:21:04.050333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f390a-3442-4c65-855f-49b7ed9066d9 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:21:04.055227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 644/1300] 10.222.0.41 () {56 vars in 1251 bytes} [Sat May 28 03:21:03 2022] GET /identity/v3/endpoints?service_id=c8d392e1566448b7a65cb37acb52295c&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:21:04.060892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:21:04.142063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:21:04.159437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dc4c2-c77a-4ed2-8ba8-ad632e993b88 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:21:04.192315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8601f6c-84b1-44d7-991d-1a76811a04e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:21:04.192970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601f6c-84b1-44d7-991d-1a76811a04e4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:04.193304 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601f6c-84b1-44d7-991d-1a76811a04e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:04.193584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601f6c-84b1-44d7-991d-1a76811a04e4 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:04.194267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8601f6c-84b1-44d7-991d-1a76811a04e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:21:04.195697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8601f6c-84b1-44d7-991d-1a76811a04e4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:21:04.202931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 657/1301] 10.222.0.41 () {56 vars in 1087 bytes} [Sat May 28 03:21:04 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:21:06.382428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ae00-ebc3-4757-be8f-8c9f439844a9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:06.383009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ae00-ebc3-4757-be8f-8c9f439844a9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:06.383009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993ae00-ebc3-4757-be8f-8c9f439844a9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:06.384346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 645/1302] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:21:06.389505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca423a89-dd59-497a-858d-515046159f05 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:06.390467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca423a89-dd59-497a-858d-515046159f05 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:06.390663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca423a89-dd59-497a-858d-515046159f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:06.406830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ca423a89-dd59-497a-858d-515046159f05 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:21:06.408002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ca423a89-dd59-497a-858d-515046159f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:21:06.472603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 658/1303] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:21:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 03:21:06.494941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993ae00-ebc3-4757-be8f-8c9f439844a9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:21:06.586788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:21:06.593285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993ae00-ebc3-4757-be8f-8c9f439844a9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:21:06.626403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:21:06.627181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:06.627456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:06.627728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:06.628366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:21:06.630048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 03:21:06.643226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 03:21:06.645048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.get_or_create( May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application with Lock( May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self._enter() May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application return self.creator() May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application created_value = creator( May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: ERROR keystone.server.flask.application May 28 03:21:06.646229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 646/1304] 10.222.0.41 () {56 vars in 1117 bytes} [Sat May 28 03:21:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 03:21:06.650597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca423a89-dd59-497a-858d-515046159f05 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:21:06.725733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:21:06.747570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca423a89-dd59-497a-858d-515046159f05 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:21:06.779735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30271953-a359-48ba-9166-20164f725ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:21:06.780407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271953-a359-48ba-9166-20164f725ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:06.780692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271953-a359-48ba-9166-20164f725ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:06.780967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271953-a359-48ba-9166-20164f725ed6 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:06.781704 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30271953-a359-48ba-9166-20164f725ed6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:21:06.782012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30271953-a359-48ba-9166-20164f725ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:21:06.783439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30271953-a359-48ba-9166-20164f725ed6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:21:06.789407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 659/1305] 10.222.0.41 () {56 vars in 1116 bytes} [Sat May 28 03:21:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:21:06.795167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:21:06.892600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7f1ae-df1e-42f3-ad3b-35c3e7da8c11 None admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:21:06.927024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556b5e7-ebb8-488e-b33c-19de1d01eef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:21:06.927668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556b5e7-ebb8-488e-b33c-19de1d01eef8 None admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:21:06.932266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556b5e7-ebb8-488e-b33c-19de1d01eef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:21:06.932266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556b5e7-ebb8-488e-b33c-19de1d01eef8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:21:06.932266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556b5e7-ebb8-488e-b33c-19de1d01eef8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:21:06.932266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556b5e7-ebb8-488e-b33c-19de1d01eef8 None admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:21:07.070871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 647/1306] 10.222.0.41 () {60 vars in 1144 bytes} [Sat May 28 03:21:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:21:16.786160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=30 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:21:17.066099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:26:52.167634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155640a2-dc01-4ffa-9198-0298f5fa8e78 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:52.168172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155640a2-dc01-4ffa-9198-0298f5fa8e78 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:52.168172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155640a2-dc01-4ffa-9198-0298f5fa8e78 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:52.170562 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 660/1307] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:26:52.175607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7799-0a95-488e-9f13-5852aa17add6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:52.176042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7799-0a95-488e-9f13-5852aa17add6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:52.176291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de7799-0a95-488e-9f13-5852aa17add6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:52.255765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-92de7799-0a95-488e-9f13-5852aa17add6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:26:52.256981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-92de7799-0a95-488e-9f13-5852aa17add6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:52.365617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 648/1308] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:26:52.433182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4737b7-379b-4f88-9b86-97b01cd583a7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:52.433487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4737b7-379b-4f88-9b86-97b01cd583a7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:52.433785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4737b7-379b-4f88-9b86-97b01cd583a7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:52.435510 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 661/1309] 10.222.0.41 () {54 vars in 964 bytes} [Sat May 28 03:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:26:52.444107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e664eb-9243-4d80-9139-cfa3429cebfd None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:52.444862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e664eb-9243-4d80-9139-cfa3429cebfd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:52.445605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e664eb-9243-4d80-9139-cfa3429cebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:52.559688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a3e664eb-9243-4d80-9139-cfa3429cebfd None None] MFA Rules not processed for user `1dd9a0e75308460080902d1da74074c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:26:52.561446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a3e664eb-9243-4d80-9139-cfa3429cebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:52.593456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:26:52.670726 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 649/1310] 10.222.0.41 () {58 vars in 1061 bytes} [Sat May 28 03:26:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:26:52.680215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4737b7-379b-4f88-9b86-97b01cd583a7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:26:52.682313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-de4737b7-379b-4f88-9b86-97b01cd583a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:52.825538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4737b7-379b-4f88-9b86-97b01cd583a7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:26:52.910685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9156b3-e1ef-4778-94ae-8a4de0db09a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:26:52.911553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9156b3-e1ef-4778-94ae-8a4de0db09a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:52.911947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9156b3-e1ef-4778-94ae-8a4de0db09a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:52.912392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9156b3-e1ef-4778-94ae-8a4de0db09a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:52.914235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0b9156b3-e1ef-4778-94ae-8a4de0db09a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:52.974200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:26:53.032728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9156b3-e1ef-4778-94ae-8a4de0db09a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:26:53.034654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9156b3-e1ef-4778-94ae-8a4de0db09a9 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:26:53.055024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 662/1311] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:26:55.547812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fda3e-1f86-4261-b35e-75a1ad88d5e2 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:55.547812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fda3e-1f86-4261-b35e-75a1ad88d5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:55.547812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fda3e-1f86-4261-b35e-75a1ad88d5e2 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:55.552306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 650/1312] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:26:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:26:55.554140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d371714-c89d-459c-9a2a-924ea203e7cd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:55.554554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d371714-c89d-459c-9a2a-924ea203e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:55.554758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d371714-c89d-459c-9a2a-924ea203e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:55.653195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5d371714-c89d-459c-9a2a-924ea203e7cd None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:26:55.654631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d371714-c89d-459c-9a2a-924ea203e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:55.786187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 663/1313] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:26:55.825726 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fda3e-1f86-4261-b35e-75a1ad88d5e2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:26:55.827310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ea4fda3e-1f86-4261-b35e-75a1ad88d5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:55.947206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fda3e-1f86-4261-b35e-75a1ad88d5e2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:26:56.025378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:26:56.027941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96081c29-dc5c-4630-9928-ba4256dc9739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:26:56.029176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96081c29-dc5c-4630-9928-ba4256dc9739 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:56.029469 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96081c29-dc5c-4630-9928-ba4256dc9739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:56.029747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96081c29-dc5c-4630-9928-ba4256dc9739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:56.031396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-96081c29-dc5c-4630-9928-ba4256dc9739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:56.150738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96081c29-dc5c-4630-9928-ba4256dc9739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:26:56.152190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96081c29-dc5c-4630-9928-ba4256dc9739 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:26:56.171819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 651/1314] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:26:58.997749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d9938-ba57-4193-8d88-fa0e38183020 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:58.998279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d9938-ba57-4193-8d88-fa0e38183020 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:58.998279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d9938-ba57-4193-8d88-fa0e38183020 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:58.999667 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 664/1315] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:26:59.005034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456d3b-d14b-47f8-ba94-32824f2826a7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:59.005477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456d3b-d14b-47f8-ba94-32824f2826a7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:59.005639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456d3b-d14b-47f8-ba94-32824f2826a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:59.071908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b7456d3b-d14b-47f8-ba94-32824f2826a7 None None] MFA Rules not processed for user `9a28cf8a3ada43b49924f24ae6da72e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:26:59.073124 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b7456d3b-d14b-47f8-ba94-32824f2826a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:59.125292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:26:59.168683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 652/1316] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:26:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:26:59.207885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d9938-ba57-4193-8d88-fa0e38183020 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:26:59.209462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-676d9938-ba57-4193-8d88-fa0e38183020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:59.219122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:26:59.314470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d9938-ba57-4193-8d88-fa0e38183020 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:26:59.387166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11759ba0-78cc-46bc-aa80-ab40715d6180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:26:59.387807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11759ba0-78cc-46bc-aa80-ab40715d6180 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:26:59.388092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11759ba0-78cc-46bc-aa80-ab40715d6180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:26:59.388368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11759ba0-78cc-46bc-aa80-ab40715d6180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:26:59.390048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-11759ba0-78cc-46bc-aa80-ab40715d6180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:26:59.455947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:26:59.501802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11759ba0-78cc-46bc-aa80-ab40715d6180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:26:59.503298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11759ba0-78cc-46bc-aa80-ab40715d6180 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:26:59.521163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 665/1317] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:26:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:02.557868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058a68-cf43-4ae6-9c94-7edb4d759ce6 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:02.558398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058a68-cf43-4ae6-9c94-7edb4d759ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:02.558493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79058a68-cf43-4ae6-9c94-7edb4d759ce6 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:02.559825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 653/1318] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:02.565033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db641db7-8a7f-4ffe-a8ac-10ede926d886 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:02.565465 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db641db7-8a7f-4ffe-a8ac-10ede926d886 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:02.565666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db641db7-8a7f-4ffe-a8ac-10ede926d886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:02.629081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-db641db7-8a7f-4ffe-a8ac-10ede926d886 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:02.630283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-db641db7-8a7f-4ffe-a8ac-10ede926d886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:02.723916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 666/1319] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:02.774644 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45980ef7-1427-4a56-b526-9eeda2bf4744 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:02.774932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45980ef7-1427-4a56-b526-9eeda2bf4744 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:02.775214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45980ef7-1427-4a56-b526-9eeda2bf4744 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:02.776589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 654/1320] 10.222.0.41 () {54 vars in 964 bytes} [Sat May 28 03:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:02.782249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6169-5fbc-4171-9090-3b4724b846f6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:02.782542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6169-5fbc-4171-9090-3b4724b846f6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:02.782822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6169-5fbc-4171-9090-3b4724b846f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:02.803179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:02.856413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-17cf6169-5fbc-4171-9090-3b4724b846f6 None None] MFA Rules not processed for user `1dd9a0e75308460080902d1da74074c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:02.857580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-17cf6169-5fbc-4171-9090-3b4724b846f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:02.933157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 667/1321] 10.222.0.41 () {58 vars in 1061 bytes} [Sat May 28 03:27:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:02.942614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45980ef7-1427-4a56-b526-9eeda2bf4744 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:02.944137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-45980ef7-1427-4a56-b526-9eeda2bf4744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:03.048384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45980ef7-1427-4a56-b526-9eeda2bf4744 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:03.120569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dd450-632b-4f4c-9a14-2390fae9d35f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:03.121166 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd450-632b-4f4c-9a14-2390fae9d35f service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:03.121453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd450-632b-4f4c-9a14-2390fae9d35f service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:03.121733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd450-632b-4f4c-9a14-2390fae9d35f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:03.123369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-194dd450-632b-4f4c-9a14-2390fae9d35f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:03.130574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:03.229568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dd450-632b-4f4c-9a14-2390fae9d35f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:03.231271 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dd450-632b-4f4c-9a14-2390fae9d35f service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:03.248729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 655/1322] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:06.974436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505629e4-f1dd-47d1-9a9a-e3bbd4be60ca None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:06.974963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505629e4-f1dd-47d1-9a9a-e3bbd4be60ca None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:06.974963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505629e4-f1dd-47d1-9a9a-e3bbd4be60ca None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:06.976352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 668/1323] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:06.981826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd93e-8214-454f-ad64-5ae41259150e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:06.982280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd93e-8214-454f-ad64-5ae41259150e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:06.982375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd93e-8214-454f-ad64-5ae41259150e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:07.045875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-91edd93e-8214-454f-ad64-5ae41259150e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:07.047097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-91edd93e-8214-454f-ad64-5ae41259150e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:07.106974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:07.151654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 656/1324] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:07.188339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505629e4-f1dd-47d1-9a9a-e3bbd4be60ca None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:07.189601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-505629e4-f1dd-47d1-9a9a-e3bbd4be60ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:07.297619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505629e4-f1dd-47d1-9a9a-e3bbd4be60ca None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:07.309847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:07.372646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7454167-e23f-49df-bdf2-0c31fbd9f791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:07.373332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7454167-e23f-49df-bdf2-0c31fbd9f791 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:07.373619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7454167-e23f-49df-bdf2-0c31fbd9f791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:07.373899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7454167-e23f-49df-bdf2-0c31fbd9f791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:07.375537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e7454167-e23f-49df-bdf2-0c31fbd9f791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:07.494038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7454167-e23f-49df-bdf2-0c31fbd9f791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:07.494038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7454167-e23f-49df-bdf2-0c31fbd9f791 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:07.509063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 669/1325] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:09.775250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b3c4-d5c4-4d69-8e90-d57207e7e2a1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:09.775806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b3c4-d5c4-4d69-8e90-d57207e7e2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:09.775806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475b3c4-d5c4-4d69-8e90-d57207e7e2a1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:09.777181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 657/1326] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:09.782483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776322f-1f9e-44ae-bde3-43360c9a8b2a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:09.782957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776322f-1f9e-44ae-bde3-43360c9a8b2a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:09.783060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776322f-1f9e-44ae-bde3-43360c9a8b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:09.830051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:09.847634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4776322f-1f9e-44ae-bde3-43360c9a8b2a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:09.848832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4776322f-1f9e-44ae-bde3-43360c9a8b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:09.948744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 670/1327] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:09.984024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475b3c4-d5c4-4d69-8e90-d57207e7e2a1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:09.985634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2475b3c4-d5c4-4d69-8e90-d57207e7e2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:10.087465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475b3c4-d5c4-4d69-8e90-d57207e7e2a1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:10.157564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3215712-c9f2-4890-9c51-1b47d9a997e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:10.158717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3215712-c9f2-4890-9c51-1b47d9a997e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:10.158995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3215712-c9f2-4890-9c51-1b47d9a997e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:10.159263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3215712-c9f2-4890-9c51-1b47d9a997e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:10.160871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b3215712-c9f2-4890-9c51-1b47d9a997e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:10.177549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:10.268362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3215712-c9f2-4890-9c51-1b47d9a997e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:10.269873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3215712-c9f2-4890-9c51-1b47d9a997e8 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:10.288472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 658/1328] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:12.536955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b17760-5b12-492e-9254-9be493c15c2e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:12.537530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b17760-5b12-492e-9254-9be493c15c2e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:12.537530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b17760-5b12-492e-9254-9be493c15c2e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:12.539056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 671/1329] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:12.544056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c12e61-3900-4115-9194-4c9c38059a1d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:12.544450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c12e61-3900-4115-9194-4c9c38059a1d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:12.544733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c12e61-3900-4115-9194-4c9c38059a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:12.608357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-12c12e61-3900-4115-9194-4c9c38059a1d None None] MFA Rules not processed for user `9a28cf8a3ada43b49924f24ae6da72e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:12.609600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-12c12e61-3900-4115-9194-4c9c38059a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:12.673139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:12.709040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 659/1330] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:12.746276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b17760-5b12-492e-9254-9be493c15c2e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:12.747802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f3b17760-5b12-492e-9254-9be493c15c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:12.850286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b17760-5b12-492e-9254-9be493c15c2e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:12.871118 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:12.928048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325b4ee-93c7-4732-b89d-c9a1f2cce1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:12.928694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325b4ee-93c7-4732-b89d-c9a1f2cce1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:12.928979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325b4ee-93c7-4732-b89d-c9a1f2cce1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:12.929299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325b4ee-93c7-4732-b89d-c9a1f2cce1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:12.930942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1325b4ee-93c7-4732-b89d-c9a1f2cce1f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:13.038863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325b4ee-93c7-4732-b89d-c9a1f2cce1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:13.040300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325b4ee-93c7-4732-b89d-c9a1f2cce1f7 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:13.058232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 672/1331] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:15.670179 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2080a-e9bb-4088-9bc9-8feb3bac3fe9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:15.670668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2080a-e9bb-4088-9bc9-8feb3bac3fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:15.670779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2080a-e9bb-4088-9bc9-8feb3bac3fe9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:15.672110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 660/1332] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:15.678300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e0b5b-7c63-4c50-9e4c-54707f2d9acd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:15.678717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e0b5b-7c63-4c50-9e4c-54707f2d9acd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:15.678817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e0b5b-7c63-4c50-9e4c-54707f2d9acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:15.741173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:15.744042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5d7e0b5b-7c63-4c50-9e4c-54707f2d9acd None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:15.745281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d7e0b5b-7c63-4c50-9e4c-54707f2d9acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:15.840945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 673/1333] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:15.877345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2080a-e9bb-4088-9bc9-8feb3bac3fe9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:15.878901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6ed2080a-e9bb-4088-9bc9-8feb3bac3fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:15.988171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2080a-e9bb-4088-9bc9-8feb3bac3fe9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:16.065318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5a8a2-9ef2-4e2d-8de0-caf534b6499e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:16.065976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a8a2-9ef2-4e2d-8de0-caf534b6499e service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:16.066266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a8a2-9ef2-4e2d-8de0-caf534b6499e service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:16.066541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5a8a2-9ef2-4e2d-8de0-caf534b6499e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:16.068227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-76d5a8a2-9ef2-4e2d-8de0-caf534b6499e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:16.097005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:16.181663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5a8a2-9ef2-4e2d-8de0-caf534b6499e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:16.183110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5a8a2-9ef2-4e2d-8de0-caf534b6499e service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:16.203552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 661/1334] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:18.467859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea1f39-3ad9-4e93-bbf6-2991ec75fc3d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:18.468372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea1f39-3ad9-4e93-bbf6-2991ec75fc3d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:18.468372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea1f39-3ad9-4e93-bbf6-2991ec75fc3d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:18.469728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 674/1335] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:18.475479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84a1a-06e3-465f-bf50-9c406b0c1465 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:18.475923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84a1a-06e3-465f-bf50-9c406b0c1465 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:18.476028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84a1a-06e3-465f-bf50-9c406b0c1465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:18.539962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-90c84a1a-06e3-465f-bf50-9c406b0c1465 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:18.541190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-90c84a1a-06e3-465f-bf50-9c406b0c1465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:18.613862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:18.635201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 662/1336] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:18.675263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea1f39-3ad9-4e93-bbf6-2991ec75fc3d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:18.676787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0aea1f39-3ad9-4e93-bbf6-2991ec75fc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:18.777837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea1f39-3ad9-4e93-bbf6-2991ec75fc3d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:18.805508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:18.849893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b07bde-a652-4ddf-84fc-f406bcbf1f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:18.850536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07bde-a652-4ddf-84fc-f406bcbf1f3c service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:18.850817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07bde-a652-4ddf-84fc-f406bcbf1f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:18.851091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b07bde-a652-4ddf-84fc-f406bcbf1f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:18.852726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-09b07bde-a652-4ddf-84fc-f406bcbf1f3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:18.959137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b07bde-a652-4ddf-84fc-f406bcbf1f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:18.960576 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b07bde-a652-4ddf-84fc-f406bcbf1f3c service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:18.978691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 675/1337] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:21.952874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b61242-abbb-4875-b2e8-2ab06f7ff4b0 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:21.953390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b61242-abbb-4875-b2e8-2ab06f7ff4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:21.953484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b61242-abbb-4875-b2e8-2ab06f7ff4b0 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:21.954842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 663/1338] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:21.959969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276c826-24e5-4f93-84f8-692eeb4c211c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:21.960390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276c826-24e5-4f93-84f8-692eeb4c211c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:21.960531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276c826-24e5-4f93-84f8-692eeb4c211c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:22.025926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3276c826-24e5-4f93-84f8-692eeb4c211c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:22.027062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3276c826-24e5-4f93-84f8-692eeb4c211c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:22.035270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:22.123539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 676/1339] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:22.158783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b61242-abbb-4875-b2e8-2ab06f7ff4b0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:22.160298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-18b61242-abbb-4875-b2e8-2ab06f7ff4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:22.267539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b61242-abbb-4875-b2e8-2ab06f7ff4b0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:22.340403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae436-5f93-41f4-aee6-90c457c4111f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:22.340961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae436-5f93-41f4-aee6-90c457c4111f service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:22.341271 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae436-5f93-41f4-aee6-90c457c4111f service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:22.341542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae436-5f93-41f4-aee6-90c457c4111f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:22.343562 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-98dae436-5f93-41f4-aee6-90c457c4111f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:22.379918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:22.452582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dae436-5f93-41f4-aee6-90c457c4111f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:22.454040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dae436-5f93-41f4-aee6-90c457c4111f service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:22.474404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 664/1340] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:26.909129 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40603-e79a-4865-a0a3-a3e978dea134 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:26.909643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40603-e79a-4865-a0a3-a3e978dea134 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:26.910188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40603-e79a-4865-a0a3-a3e978dea134 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:26.911079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 677/1341] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:26.916482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d2b0-88d8-4a14-bc52-5dc0f5bf7463 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:26.916893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d2b0-88d8-4a14-bc52-5dc0f5bf7463 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:26.917052 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00d2b0-88d8-4a14-bc52-5dc0f5bf7463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:26.964012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9b00d2b0-88d8-4a14-bc52-5dc0f5bf7463 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:26.965161 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9b00d2b0-88d8-4a14-bc52-5dc0f5bf7463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:27.042404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 665/1342] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:27.081307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc40603-e79a-4865-a0a3-a3e978dea134 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:27.083433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2fc40603-e79a-4865-a0a3-a3e978dea134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:27.118215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc40603-e79a-4865-a0a3-a3e978dea134 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:27.121742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74bf2e-5d63-430f-be55-f1465b211b4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:27.122381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74bf2e-5d63-430f-be55-f1465b211b4d service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:27.122666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74bf2e-5d63-430f-be55-f1465b211b4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:27.122943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74bf2e-5d63-430f-be55-f1465b211b4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:27.132254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74bf2e-5d63-430f-be55-f1465b211b4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:27.133738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74bf2e-5d63-430f-be55-f1465b211b4d service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:27.143162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 678/1343] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:30.426212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd7e38-1344-4997-8e22-177b585e3b38 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:30.426774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd7e38-1344-4997-8e22-177b585e3b38 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:30.426774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd7e38-1344-4997-8e22-177b585e3b38 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:30.428120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 666/1344] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:30.433532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c33122-e3fb-47d8-acc1-33b5760f6385 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:30.433945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c33122-e3fb-47d8-acc1-33b5760f6385 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:30.434126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c33122-e3fb-47d8-acc1-33b5760f6385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:30.463088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-20c33122-e3fb-47d8-acc1-33b5760f6385 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:30.464204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-20c33122-e3fb-47d8-acc1-33b5760f6385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:30.471989 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 679/1345] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:30.508658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd7e38-1344-4997-8e22-177b585e3b38 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:30.520601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd7e38-1344-4997-8e22-177b585e3b38 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:30.524202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aee62d-491d-4240-8465-ffaced9283f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:30.525044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee62d-491d-4240-8465-ffaced9283f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:30.525371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee62d-491d-4240-8465-ffaced9283f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:30.525644 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aee62d-491d-4240-8465-ffaced9283f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:30.534922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aee62d-491d-4240-8465-ffaced9283f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:30.538419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aee62d-491d-4240-8465-ffaced9283f8 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:30.545897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 667/1346] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:35.084487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8d7de-043e-48d4-bd6d-8b59eaec0112 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:35.085055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8d7de-043e-48d4-bd6d-8b59eaec0112 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:35.085055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8d7de-043e-48d4-bd6d-8b59eaec0112 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:35.086445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 680/1347] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:35.091875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1249343-82c6-4603-8734-8fa468ac0a91 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:35.092316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1249343-82c6-4603-8734-8fa468ac0a91 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:35.092633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1249343-82c6-4603-8734-8fa468ac0a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:35.111901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f1249343-82c6-4603-8734-8fa468ac0a91 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:35.113074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f1249343-82c6-4603-8734-8fa468ac0a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:35.120780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 668/1348] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:35.158567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8d7de-043e-48d4-bd6d-8b59eaec0112 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:35.168956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8d7de-043e-48d4-bd6d-8b59eaec0112 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:35.172431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9fb4a-36dc-48e2-97a0-158e8090fad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:35.173680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9fb4a-36dc-48e2-97a0-158e8090fad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:35.173963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9fb4a-36dc-48e2-97a0-158e8090fad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:35.174239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9fb4a-36dc-48e2-97a0-158e8090fad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:35.183140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9fb4a-36dc-48e2-97a0-158e8090fad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:35.184554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9fb4a-36dc-48e2-97a0-158e8090fad6 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:35.193938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 681/1349] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:40.660500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6293-5afc-4fa0-a3f3-0275962d9cad None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:40.661041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6293-5afc-4fa0-a3f3-0275962d9cad None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:40.661041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b6293-5afc-4fa0-a3f3-0275962d9cad None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:40.662501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 669/1350] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:40.667916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e20d3d-794f-4016-a2a5-40d1114fb042 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:40.668332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e20d3d-794f-4016-a2a5-40d1114fb042 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:40.668492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e20d3d-794f-4016-a2a5-40d1114fb042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:40.796708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-55e20d3d-794f-4016-a2a5-40d1114fb042 None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:40.798469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-55e20d3d-794f-4016-a2a5-40d1114fb042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:40.845509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:40.909045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 682/1351] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:27:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:27:44.950810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455c07-139e-4b43-abbd-afca9c2ccede None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:44.951354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455c07-139e-4b43-abbd-afca9c2ccede None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:44.951354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2455c07-139e-4b43-abbd-afca9c2ccede None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:44.952915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 670/1352] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:44.958187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb04bd4-fcb9-4112-8c05-e79f0977dd4f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:44.958635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb04bd4-fcb9-4112-8c05-e79f0977dd4f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:44.958751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb04bd4-fcb9-4112-8c05-e79f0977dd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:44.977171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8eb04bd4-fcb9-4112-8c05-e79f0977dd4f None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:44.978370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8eb04bd4-fcb9-4112-8c05-e79f0977dd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:44.987702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 683/1353] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:27:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:45.105693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=93 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:46.475930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897301d7-f698-400a-97a6-cd784951b6fc None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:46.476511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897301d7-f698-400a-97a6-cd784951b6fc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:46.476511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897301d7-f698-400a-97a6-cd784951b6fc None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:46.477797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 671/1354] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:46.483030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9787db-b2a2-4069-b26d-123bb510c726 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:46.483478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9787db-b2a2-4069-b26d-123bb510c726 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:46.483840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9787db-b2a2-4069-b26d-123bb510c726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:46.503045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5b9787db-b2a2-4069-b26d-123bb510c726 None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:46.504219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5b9787db-b2a2-4069-b26d-123bb510c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:46.512571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 684/1355] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:27:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:50.381730 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f8421-f44b-4af8-972a-1add06a10264 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:50.382298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f8421-f44b-4af8-972a-1add06a10264 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:50.382298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f8421-f44b-4af8-972a-1add06a10264 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:50.383898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 672/1356] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:50.389216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b023d94-d023-4c94-9355-0d65a6c350af None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:50.389637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b023d94-d023-4c94-9355-0d65a6c350af None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:50.389785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b023d94-d023-4c94-9355-0d65a6c350af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:50.409450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7b023d94-d023-4c94-9355-0d65a6c350af None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:50.410677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7b023d94-d023-4c94-9355-0d65a6c350af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:50.418769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 685/1357] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:50.481945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9d827-9fd8-4a2c-96f9-4ddda8317a2d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:50.482586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9d827-9fd8-4a2c-96f9-4ddda8317a2d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:50.483081 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9d827-9fd8-4a2c-96f9-4ddda8317a2d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:50.485273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 673/1358] 10.222.0.41 () {54 vars in 961 bytes} [Sat May 28 03:27:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:50.494491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9671bb-9c09-480c-9bb0-27b60bd8e132 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:50.495098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9671bb-9c09-480c-9bb0-27b60bd8e132 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:50.495679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9671bb-9c09-480c-9bb0-27b60bd8e132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:50.550832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4a9671bb-9c09-480c-9bb0-27b60bd8e132 None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:50.552494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4a9671bb-9c09-480c-9bb0-27b60bd8e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:50.645661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 686/1359] 10.222.0.41 () {58 vars in 1058 bytes} [Sat May 28 03:27:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:27:50.653393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9d827-9fd8-4a2c-96f9-4ddda8317a2d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:50.670618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9d827-9fd8-4a2c-96f9-4ddda8317a2d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:50.677523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34193f8a-1a6b-44b1-a4b1-cb320f1af741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:50.678389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193f8a-1a6b-44b1-a4b1-cb320f1af741 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:50.678965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193f8a-1a6b-44b1-a4b1-cb320f1af741 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:50.679537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34193f8a-1a6b-44b1-a4b1-cb320f1af741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:50.689303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34193f8a-1a6b-44b1-a4b1-cb320f1af741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:50.692110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34193f8a-1a6b-44b1-a4b1-cb320f1af741 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:50.703011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 674/1360] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:53.139016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1c756-47d7-4f15-b3c8-425de07c6ee7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:53.139644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1c756-47d7-4f15-b3c8-425de07c6ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:53.139644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1c756-47d7-4f15-b3c8-425de07c6ee7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:53.141008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 687/1361] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:53.146559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8a420-0a8b-4e4b-8249-7e60eb8df4b3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:53.146967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8a420-0a8b-4e4b-8249-7e60eb8df4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:53.147113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8a420-0a8b-4e4b-8249-7e60eb8df4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:53.164961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-84c8a420-0a8b-4e4b-8249-7e60eb8df4b3 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:53.166167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-84c8a420-0a8b-4e4b-8249-7e60eb8df4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:53.174147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 675/1362] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:53.217827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1c756-47d7-4f15-b3c8-425de07c6ee7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:53.232540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1c756-47d7-4f15-b3c8-425de07c6ee7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:53.238717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62046d-e072-4653-8f3e-0b166d229818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:53.239422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62046d-e072-4653-8f3e-0b166d229818 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:53.239778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62046d-e072-4653-8f3e-0b166d229818 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:53.240117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62046d-e072-4653-8f3e-0b166d229818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:53.248598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62046d-e072-4653-8f3e-0b166d229818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:53.250267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62046d-e072-4653-8f3e-0b166d229818 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:53.260567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 688/1363] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:55.936403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ba5a-b1c1-4437-ac35-5214537df9d1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:55.937288 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ba5a-b1c1-4437-ac35-5214537df9d1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:55.937572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ba5a-b1c1-4437-ac35-5214537df9d1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:55.940728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 676/1364] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:55.944534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6ab6-bb42-45ec-a8f9-822474824d73 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:55.949453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6ab6-bb42-45ec-a8f9-822474824d73 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:55.949932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6ab6-bb42-45ec-a8f9-822474824d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:55.970693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3b6f6ab6-bb42-45ec-a8f9-822474824d73 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:55.972133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3b6f6ab6-bb42-45ec-a8f9-822474824d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:55.980318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 689/1365] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:56.026435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6f785-48f0-4727-b1ee-9cabac1b3dbb None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:56.026690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6f785-48f0-4727-b1ee-9cabac1b3dbb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:56.026969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6f785-48f0-4727-b1ee-9cabac1b3dbb None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:56.028325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 677/1366] 10.222.0.41 () {54 vars in 961 bytes} [Sat May 28 03:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:56.034256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bac30-2dd1-4708-999a-a89957c8bf3b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:56.034552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bac30-2dd1-4708-999a-a89957c8bf3b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:56.034836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bac30-2dd1-4708-999a-a89957c8bf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:56.053227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c38bac30-2dd1-4708-999a-a89957c8bf3b None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:56.054398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c38bac30-2dd1-4708-999a-a89957c8bf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:56.063556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 690/1367] 10.222.0.41 () {58 vars in 1058 bytes} [Sat May 28 03:27:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:56.071510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6f785-48f0-4727-b1ee-9cabac1b3dbb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:56.085472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6f785-48f0-4727-b1ee-9cabac1b3dbb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:56.091034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89fdb1-b54d-43ba-825a-1af014fc1d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:56.091666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89fdb1-b54d-43ba-825a-1af014fc1d58 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:56.091952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89fdb1-b54d-43ba-825a-1af014fc1d58 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:56.092234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89fdb1-b54d-43ba-825a-1af014fc1d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:56.100885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89fdb1-b54d-43ba-825a-1af014fc1d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:56.102496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89fdb1-b54d-43ba-825a-1af014fc1d58 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:56.112672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 678/1368] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:58.530839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe937f4-cdfc-4837-97e7-75c9b943f2e3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:58.532524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe937f4-cdfc-4837-97e7-75c9b943f2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:58.533343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe937f4-cdfc-4837-97e7-75c9b943f2e3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:58.536014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 691/1369] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:58.541027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b3b5b-d3a1-4ef2-811e-226586e44bfe None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:58.541862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b3b5b-d3a1-4ef2-811e-226586e44bfe None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:58.542294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30b3b5b-d3a1-4ef2-811e-226586e44bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:58.561949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e30b3b5b-d3a1-4ef2-811e-226586e44bfe None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:58.563534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e30b3b5b-d3a1-4ef2-811e-226586e44bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:58.572888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 679/1370] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:27:58.619676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe937f4-cdfc-4837-97e7-75c9b943f2e3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:58.634670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe937f4-cdfc-4837-97e7-75c9b943f2e3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:58.641541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfbacb-12f7-4736-9263-e4e449b28936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:58.642383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfbacb-12f7-4736-9263-e4e449b28936 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:58.642819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfbacb-12f7-4736-9263-e4e449b28936 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:58.643244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfbacb-12f7-4736-9263-e4e449b28936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:58.654212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfbacb-12f7-4736-9263-e4e449b28936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:58.656028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfbacb-12f7-4736-9263-e4e449b28936 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:58.666003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9c251-d0b2-4738-8eee-e3457efea930 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:58.666290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9c251-d0b2-4738-8eee-e3457efea930 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:58.666747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9c251-d0b2-4738-8eee-e3457efea930 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:58.668275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 680/1371] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:27:58.669544 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 692/1372] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:27:58.673797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788cc7-cc1c-4267-a0eb-bd9cb745af0a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:58.674086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788cc7-cc1c-4267-a0eb-bd9cb745af0a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:58.674371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788cc7-cc1c-4267-a0eb-bd9cb745af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:58.693327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7e788cc7-cc1c-4267-a0eb-bd9cb745af0a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:58.697690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7e788cc7-cc1c-4267-a0eb-bd9cb745af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:58.703009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 693/1373] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:27:59.008046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceca500-46a7-46a9-a3b6-f673b3e5476a None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:59.008337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceca500-46a7-46a9-a3b6-f673b3e5476a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:59.009171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceca500-46a7-46a9-a3b6-f673b3e5476a None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:59.010329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 681/1374] 10.222.0.41 () {54 vars in 963 bytes} [Sat May 28 03:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:27:59.017769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe73877-617c-4386-9396-89f74bb232cf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:59.018072 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe73877-617c-4386-9396-89f74bb232cf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:59.018326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe73877-617c-4386-9396-89f74bb232cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:59.026071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:27:59.075121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bfe73877-617c-4386-9396-89f74bb232cf None None] MFA Rules not processed for user `7092a719870d4b5eafec8d4c51226b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:27:59.076340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bfe73877-617c-4386-9396-89f74bb232cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:27:59.139658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 694/1375] 10.222.0.41 () {58 vars in 1060 bytes} [Sat May 28 03:27:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:27:59.147617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceca500-46a7-46a9-a3b6-f673b3e5476a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:27:59.158182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceca500-46a7-46a9-a3b6-f673b3e5476a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:27:59.161689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01a453-1001-43aa-95a9-b4f3056eda49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:27:59.162318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01a453-1001-43aa-95a9-b4f3056eda49 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:27:59.162595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01a453-1001-43aa-95a9-b4f3056eda49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:27:59.163093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01a453-1001-43aa-95a9-b4f3056eda49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:27:59.172443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01a453-1001-43aa-95a9-b4f3056eda49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:27:59.173904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01a453-1001-43aa-95a9-b4f3056eda49 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:27:59.183547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 682/1376] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:01.061135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ef39b-1853-4f97-92d3-fc35b084f0e5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:01.061689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ef39b-1853-4f97-92d3-fc35b084f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:01.061689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ef39b-1853-4f97-92d3-fc35b084f0e5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:01.063166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 695/1377] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:01.068960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126acf9-52b2-425d-b4d8-88fda2e816ef None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:01.069531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126acf9-52b2-425d-b4d8-88fda2e816ef None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:01.069623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126acf9-52b2-425d-b4d8-88fda2e816ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:01.088908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1126acf9-52b2-425d-b4d8-88fda2e816ef None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:01.090178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1126acf9-52b2-425d-b4d8-88fda2e816ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:01.099226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 683/1378] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:01.144475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ef39b-1853-4f97-92d3-fc35b084f0e5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:01.159202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ef39b-1853-4f97-92d3-fc35b084f0e5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:01.164789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60cfbd6-8744-4423-87af-6143a63a0c37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:01.165461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cfbd6-8744-4423-87af-6143a63a0c37 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:01.165747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cfbd6-8744-4423-87af-6143a63a0c37 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:01.166024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60cfbd6-8744-4423-87af-6143a63a0c37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:01.174825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cfbd6-8744-4423-87af-6143a63a0c37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:01.176411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60cfbd6-8744-4423-87af-6143a63a0c37 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:01.186231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 696/1379] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:03.548769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2264cbf-f9bf-40f3-ad6e-a03bc8596d68 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:03.549705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2264cbf-f9bf-40f3-ad6e-a03bc8596d68 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:03.549993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2264cbf-f9bf-40f3-ad6e-a03bc8596d68 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:03.551374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 684/1380] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:03.557070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce317a3c-e3cf-45da-a7a1-e77ec3c77604 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:03.557546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce317a3c-e3cf-45da-a7a1-e77ec3c77604 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:03.557728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce317a3c-e3cf-45da-a7a1-e77ec3c77604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:03.576017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ce317a3c-e3cf-45da-a7a1-e77ec3c77604 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:03.577480 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ce317a3c-e3cf-45da-a7a1-e77ec3c77604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:03.586082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 697/1381] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:03.630652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2264cbf-f9bf-40f3-ad6e-a03bc8596d68 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:03.645364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2264cbf-f9bf-40f3-ad6e-a03bc8596d68 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:03.650968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931750e-273e-48cc-aea1-0aa94e310075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:03.651699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931750e-273e-48cc-aea1-0aa94e310075 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:03.652059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931750e-273e-48cc-aea1-0aa94e310075 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:03.652512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931750e-273e-48cc-aea1-0aa94e310075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:03.661895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931750e-273e-48cc-aea1-0aa94e310075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:03.663665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931750e-273e-48cc-aea1-0aa94e310075 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:03.675794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 685/1382] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:06.130256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cdd67-458c-4d63-9ca5-bec1424b5bb3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:06.130780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cdd67-458c-4d63-9ca5-bec1424b5bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:06.130861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cdd67-458c-4d63-9ca5-bec1424b5bb3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:06.132394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 698/1383] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:06.138000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c413c94-bb21-4e55-bdae-8adb0582d7b8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:06.138454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c413c94-bb21-4e55-bdae-8adb0582d7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:06.138571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c413c94-bb21-4e55-bdae-8adb0582d7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:06.157394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8c413c94-bb21-4e55-bdae-8adb0582d7b8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:06.158779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8c413c94-bb21-4e55-bdae-8adb0582d7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:06.169159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 686/1384] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:06.208245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cdd67-458c-4d63-9ca5-bec1424b5bb3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:06.224220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cdd67-458c-4d63-9ca5-bec1424b5bb3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:06.230059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd42641d-c11b-4387-9565-8adb1367c009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:06.230792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42641d-c11b-4387-9565-8adb1367c009 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:06.231150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42641d-c11b-4387-9565-8adb1367c009 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:06.231887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd42641d-c11b-4387-9565-8adb1367c009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:06.241359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42641d-c11b-4387-9565-8adb1367c009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:06.243082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd42641d-c11b-4387-9565-8adb1367c009 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:06.253840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 699/1385] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:08.470896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf77123-7a7c-45f7-8ec1-fe28f6c1d467 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:08.471595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf77123-7a7c-45f7-8ec1-fe28f6c1d467 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:08.471595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf77123-7a7c-45f7-8ec1-fe28f6c1d467 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:08.472756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 687/1386] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:08.478332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61721c7-6310-4ff6-8bfc-7c3bd66ec2ec None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:08.478825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61721c7-6310-4ff6-8bfc-7c3bd66ec2ec None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:08.478916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61721c7-6310-4ff6-8bfc-7c3bd66ec2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:08.497262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c61721c7-6310-4ff6-8bfc-7c3bd66ec2ec None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:08.498458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c61721c7-6310-4ff6-8bfc-7c3bd66ec2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:08.507182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 700/1387] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:08.551583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf77123-7a7c-45f7-8ec1-fe28f6c1d467 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:08.566713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf77123-7a7c-45f7-8ec1-fe28f6c1d467 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:08.572181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c763b-401c-4f8e-8506-21211e223210 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:08.572823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c763b-401c-4f8e-8506-21211e223210 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:08.573127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c763b-401c-4f8e-8506-21211e223210 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:08.573421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c763b-401c-4f8e-8506-21211e223210 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:08.582031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c763b-401c-4f8e-8506-21211e223210 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:08.583638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c763b-401c-4f8e-8506-21211e223210 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:08.593285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 688/1388] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:11.296684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d983f-b8dd-4712-8a5c-a28392162aea None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:11.297299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d983f-b8dd-4712-8a5c-a28392162aea None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:11.297299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d983f-b8dd-4712-8a5c-a28392162aea None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:11.298711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 701/1389] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:11.304931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b3f16-0271-46cb-ba87-847d72e5447b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:11.305393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b3f16-0271-46cb-ba87-847d72e5447b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:11.305766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b3f16-0271-46cb-ba87-847d72e5447b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:11.327451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8a8b3f16-0271-46cb-ba87-847d72e5447b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:11.329606 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8a8b3f16-0271-46cb-ba87-847d72e5447b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:11.339525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 689/1390] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:11.385973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d983f-b8dd-4712-8a5c-a28392162aea None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:11.400819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d983f-b8dd-4712-8a5c-a28392162aea None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:11.406338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba38b6b-6483-4865-af9c-b84eb7309466 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:11.407506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba38b6b-6483-4865-af9c-b84eb7309466 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:11.407795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba38b6b-6483-4865-af9c-b84eb7309466 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:11.408075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba38b6b-6483-4865-af9c-b84eb7309466 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:11.416308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba38b6b-6483-4865-af9c-b84eb7309466 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:11.418004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba38b6b-6483-4865-af9c-b84eb7309466 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:11.427554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 702/1391] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:13.875880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa042b10-bcf9-432c-ba9d-af2de857aa74 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:13.876503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa042b10-bcf9-432c-ba9d-af2de857aa74 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:13.876503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa042b10-bcf9-432c-ba9d-af2de857aa74 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:13.877835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 690/1392] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:13.883157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598686b-31cf-41e3-a438-d69b24252558 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:13.883583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598686b-31cf-41e3-a438-d69b24252558 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:13.883713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598686b-31cf-41e3-a438-d69b24252558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:13.901394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4598686b-31cf-41e3-a438-d69b24252558 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:13.902656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4598686b-31cf-41e3-a438-d69b24252558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:13.911341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 703/1393] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:13.954410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa042b10-bcf9-432c-ba9d-af2de857aa74 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:13.968611 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa042b10-bcf9-432c-ba9d-af2de857aa74 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:13.974559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056a5ee-862e-4cf6-8715-ad869353963f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:13.975193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056a5ee-862e-4cf6-8715-ad869353963f service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:13.975482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056a5ee-862e-4cf6-8715-ad869353963f service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:13.975761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056a5ee-862e-4cf6-8715-ad869353963f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:13.984104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056a5ee-862e-4cf6-8715-ad869353963f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:13.986263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056a5ee-862e-4cf6-8715-ad869353963f service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:13.995458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 691/1394] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:15.122825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19029-065e-42ec-84f5-b6758308f87c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:15.123410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19029-065e-42ec-84f5-b6758308f87c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:15.123410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19029-065e-42ec-84f5-b6758308f87c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:15.124765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 704/1395] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:28:15.130168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413edfe-90b3-4eb5-91bf-2fc163c59570 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:15.130599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413edfe-90b3-4eb5-91bf-2fc163c59570 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:15.130708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413edfe-90b3-4eb5-91bf-2fc163c59570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:15.143297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=54 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:28:15.149220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0413edfe-90b3-4eb5-91bf-2fc163c59570 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:15.151589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0413edfe-90b3-4eb5-91bf-2fc163c59570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:15.159231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 692/1396] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:15.216594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712d58-016e-4c18-ba19-db96ef40dba8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:15.216957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712d58-016e-4c18-ba19-db96ef40dba8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:15.217341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b712d58-016e-4c18-ba19-db96ef40dba8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:15.219441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 705/1397] 10.222.0.41 () {54 vars in 962 bytes} [Sat May 28 03:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:15.224651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67256a-d3d6-4d0d-993d-ae5719fec076 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:15.225021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67256a-d3d6-4d0d-993d-ae5719fec076 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:15.225411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67256a-d3d6-4d0d-993d-ae5719fec076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:15.272928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0c67256a-d3d6-4d0d-993d-ae5719fec076 None None] MFA Rules not processed for user `a47589bf9e7c4acba200979e3a91a6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:15.274133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0c67256a-d3d6-4d0d-993d-ae5719fec076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:15.335109 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 693/1398] 10.222.0.41 () {58 vars in 1059 bytes} [Sat May 28 03:28:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:15.342919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b712d58-016e-4c18-ba19-db96ef40dba8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:15.354015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b712d58-016e-4c18-ba19-db96ef40dba8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:15.357806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c371bd-2f7e-4fe1-995d-f596568a4541 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47589bf9e7c4acba200979e3a91a6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:15.358431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c371bd-2f7e-4fe1-995d-f596568a4541 service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:15.358713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c371bd-2f7e-4fe1-995d-f596568a4541 service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:15.358987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c371bd-2f7e-4fe1-995d-f596568a4541 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:15.367837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c371bd-2f7e-4fe1-995d-f596568a4541 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:15.369293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c371bd-2f7e-4fe1-995d-f596568a4541 service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:15.379408 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 706/1399] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:28:16.068564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e78c5f-52c1-4f5a-a56c-dd0562d2dbe9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:16.068856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e78c5f-52c1-4f5a-a56c-dd0562d2dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:16.071673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e78c5f-52c1-4f5a-a56c-dd0562d2dbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:16.126103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-96e78c5f-52c1-4f5a-a56c-dd0562d2dbe9 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:16.127212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-96e78c5f-52c1-4f5a-a56c-dd0562d2dbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:16.189874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 694/1400] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:28:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:16.231191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e04d-0f25-4ad1-8b23-874c4b81ca66 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:16.232063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e04d-0f25-4ad1-8b23-874c4b81ca66 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:16.232063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3e04d-0f25-4ad1-8b23-874c4b81ca66 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:16.233529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 707/1401] 10.222.0.41 () {54 vars in 942 bytes} [Sat May 28 03:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:16.241481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7f8e-d4b9-466f-aabd-25cd8d7a0dc0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:16.241771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7f8e-d4b9-466f-aabd-25cd8d7a0dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:16.242073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b7f8e-d4b9-466f-aabd-25cd8d7a0dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:16.290592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d84b7f8e-d4b9-466f-aabd-25cd8d7a0dc0 None None] MFA Rules not processed for user `f2f14380f60f4fa18ed6a9579cfb071b`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:16.291765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d84b7f8e-d4b9-466f-aabd-25cd8d7a0dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:16.310098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:28:16.361505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 695/1402] 10.222.0.41 () {58 vars in 1039 bytes} [Sat May 28 03:28:16 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:16.371180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3e04d-0f25-4ad1-8b23-874c4b81ca66 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:16.384445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3e04d-0f25-4ad1-8b23-874c4b81ca66 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:16.389412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9622d6-171e-42c3-ad94-52dce8ea8e6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:16.390132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9622d6-171e-42c3-ad94-52dce8ea8e6b service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:16.390486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9622d6-171e-42c3-ad94-52dce8ea8e6b service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:16.390769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9622d6-171e-42c3-ad94-52dce8ea8e6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:16.398821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9622d6-171e-42c3-ad94-52dce8ea8e6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:16.400398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9622d6-171e-42c3-ad94-52dce8ea8e6b service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:16.408870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 708/1403] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:28:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:28:16.421139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0af4d-70d1-41a3-8873-5a0d9cb9828f None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:16.421388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0af4d-70d1-41a3-8873-5a0d9cb9828f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:16.421741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0af4d-70d1-41a3-8873-5a0d9cb9828f None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:16.423598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 696/1404] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:16.429886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3013-2891-47f8-b1ff-514c00a7b54e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:16.430189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3013-2891-47f8-b1ff-514c00a7b54e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:16.430478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c3013-2891-47f8-b1ff-514c00a7b54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:16.448535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-452c3013-2891-47f8-b1ff-514c00a7b54e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:16.449828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-452c3013-2891-47f8-b1ff-514c00a7b54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:16.458636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 709/1405] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:16.502422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0af4d-70d1-41a3-8873-5a0d9cb9828f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:16.520967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0af4d-70d1-41a3-8873-5a0d9cb9828f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:16.524255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d87f69-598e-4160-8014-13dba34c3f51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:16.524905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87f69-598e-4160-8014-13dba34c3f51 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:16.525219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87f69-598e-4160-8014-13dba34c3f51 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:16.525501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87f69-598e-4160-8014-13dba34c3f51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:16.534792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d87f69-598e-4160-8014-13dba34c3f51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:16.536376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d87f69-598e-4160-8014-13dba34c3f51 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:16.545778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 697/1406] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:18.962269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7127-06d5-4549-ad52-639f4c4891d5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:18.962855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7127-06d5-4549-ad52-639f4c4891d5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:18.962855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7127-06d5-4549-ad52-639f4c4891d5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:18.965143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 710/1407] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:18.970275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aabd7-8751-4d97-8a51-aaaea92d70b7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:18.970781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aabd7-8751-4d97-8a51-aaaea92d70b7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:18.970860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aabd7-8751-4d97-8a51-aaaea92d70b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:18.990451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-991aabd7-8751-4d97-8a51-aaaea92d70b7 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:18.991708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-991aabd7-8751-4d97-8a51-aaaea92d70b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:19.000403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 698/1408] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:19.044830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b7127-06d5-4549-ad52-639f4c4891d5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:19.059549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b7127-06d5-4549-ad52-639f4c4891d5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:19.065203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ce525-ace5-47e5-a435-7dbb4031cace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:19.065838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ce525-ace5-47e5-a435-7dbb4031cace service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:19.066124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ce525-ace5-47e5-a435-7dbb4031cace service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:19.066399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ce525-ace5-47e5-a435-7dbb4031cace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:19.076085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ce525-ace5-47e5-a435-7dbb4031cace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:19.077739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ce525-ace5-47e5-a435-7dbb4031cace service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:19.088267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 711/1409] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:29.350486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=31 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:28:29.353789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=84 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:28:40.786396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72a8f0-2ad1-4d71-991a-78fc3759b0e4 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:40.786986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72a8f0-2ad1-4d71-991a-78fc3759b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:40.787102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e72a8f0-2ad1-4d71-991a-78fc3759b0e4 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:40.788416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 699/1410] 10.222.0.41 () {54 vars in 918 bytes} [Sat May 28 03:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:40.793653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc195-7a43-4a64-9385-b68c834c3807 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:40.794115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc195-7a43-4a64-9385-b68c834c3807 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:40.794220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbc195-7a43-4a64-9385-b68c834c3807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:40.812429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4dbbc195-7a43-4a64-9385-b68c834c3807 None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:40.816266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4dbbc195-7a43-4a64-9385-b68c834c3807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:40.822265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 712/1411] 10.222.0.41 () {58 vars in 1015 bytes} [Sat May 28 03:28:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:40.836735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92eb61-6b4e-4878-a38e-ecd115c78681 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:40.837246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92eb61-6b4e-4878-a38e-ecd115c78681 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:40.837464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92eb61-6b4e-4878-a38e-ecd115c78681 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:40.838938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 700/1412] 10.222.0.41 () {54 vars in 918 bytes} [Sat May 28 03:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:40.844137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a55544-7d37-41de-b67c-7008ec503936 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:40.844433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a55544-7d37-41de-b67c-7008ec503936 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:40.844714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a55544-7d37-41de-b67c-7008ec503936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:40.862528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-93a55544-7d37-41de-b67c-7008ec503936 None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:40.867741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-93a55544-7d37-41de-b67c-7008ec503936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:40.872774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 713/1413] 10.222.0.41 () {58 vars in 1015 bytes} [Sat May 28 03:28:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:43.525360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1941-5499-476e-9436-753841adb208 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:43.532894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1941-5499-476e-9436-753841adb208 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:43.532894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff1941-5499-476e-9436-753841adb208 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:43.532894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:43.533707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886672-8e77-42f2-a67c-e91fbd73ad7f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:43.534094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886672-8e77-42f2-a67c-e91fbd73ad7f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:43.534273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0886672-8e77-42f2-a67c-e91fbd73ad7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:43.551687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c0886672-8e77-42f2-a67c-e91fbd73ad7f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:43.552879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c0886672-8e77-42f2-a67c-e91fbd73ad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:43.560674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:43.603298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff1941-5499-476e-9436-753841adb208 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:43.617677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff1941-5499-476e-9436-753841adb208 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:43.623048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aaa232-9acb-4387-a7a0-4caad8245c52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:43.623677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa232-9acb-4387-a7a0-4caad8245c52 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:43.623956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa232-9acb-4387-a7a0-4caad8245c52 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:43.624231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aaa232-9acb-4387-a7a0-4caad8245c52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:43.632366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aaa232-9acb-4387-a7a0-4caad8245c52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:43.633989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aaa232-9acb-4387-a7a0-4caad8245c52 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:43.643738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 702/1416] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:44.599453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4b11-db52-42c6-bd26-9cae6680a254 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:44.600019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4b11-db52-42c6-bd26-9cae6680a254 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:44.600019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a4b11-db52-42c6-bd26-9cae6680a254 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:44.601446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:44.606369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965d652-64c6-4c0a-b14f-6e3c92d20908 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:44.606758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965d652-64c6-4c0a-b14f-6e3c92d20908 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:44.607187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965d652-64c6-4c0a-b14f-6e3c92d20908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:44.625316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0965d652-64c6-4c0a-b14f-6e3c92d20908 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:44.626491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0965d652-64c6-4c0a-b14f-6e3c92d20908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:44.634181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:44.794127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a4b11-db52-42c6-bd26-9cae6680a254 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:44.808807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a4b11-db52-42c6-bd26-9cae6680a254 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:44.814208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0afd8-5b76-4d5d-b9fa-4b8d717fcbd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:44.814841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0afd8-5b76-4d5d-b9fa-4b8d717fcbd8 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:44.815143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0afd8-5b76-4d5d-b9fa-4b8d717fcbd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:44.815418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0afd8-5b76-4d5d-b9fa-4b8d717fcbd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:44.823477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0afd8-5b76-4d5d-b9fa-4b8d717fcbd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:44.825222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0afd8-5b76-4d5d-b9fa-4b8d717fcbd8 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:44.834782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 716/1419] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:28:45.752472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158288-d4fa-4632-aeac-735a53fd4ff5 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:45.753044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158288-d4fa-4632-aeac-735a53fd4ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:45.753044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158288-d4fa-4632-aeac-735a53fd4ff5 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:45.754460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:45.759303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035ffdb-77e5-43de-ad40-085d8174b0f1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:45.759706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035ffdb-77e5-43de-ad40-085d8174b0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:45.759858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035ffdb-77e5-43de-ad40-085d8174b0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:45.777338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9035ffdb-77e5-43de-ad40-085d8174b0f1 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:45.778505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9035ffdb-77e5-43de-ad40-085d8174b0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:45.786116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:45.941021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158288-d4fa-4632-aeac-735a53fd4ff5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:45.954786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158288-d4fa-4632-aeac-735a53fd4ff5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:45.960035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd26a84-eb10-476b-aa56-2e4c1c5f774f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:45.960657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd26a84-eb10-476b-aa56-2e4c1c5f774f service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:45.960936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd26a84-eb10-476b-aa56-2e4c1c5f774f service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:45.961249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd26a84-eb10-476b-aa56-2e4c1c5f774f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:45.969066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd26a84-eb10-476b-aa56-2e4c1c5f774f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:45.970665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd26a84-eb10-476b-aa56-2e4c1c5f774f service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:45.979844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 705/1422] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:46.891563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a7454-f194-4a51-9f0e-460a1527ae59 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:46.892182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a7454-f194-4a51-9f0e-460a1527ae59 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:46.892182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a7454-f194-4a51-9f0e-460a1527ae59 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:46.893505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:46.898511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f2d52-c5a7-4712-b0fd-4fd73ad398ae None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:46.898926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f2d52-c5a7-4712-b0fd-4fd73ad398ae None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:46.899063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f2d52-c5a7-4712-b0fd-4fd73ad398ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:46.916839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f24f2d52-c5a7-4712-b0fd-4fd73ad398ae None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:46.918057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f24f2d52-c5a7-4712-b0fd-4fd73ad398ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:46.925860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:47.073424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a7454-f194-4a51-9f0e-460a1527ae59 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:47.088194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a7454-f194-4a51-9f0e-460a1527ae59 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:47.093556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad27ab1-9519-4927-ab2b-7bd25ff21c8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:47.094184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad27ab1-9519-4927-ab2b-7bd25ff21c8e service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:47.094470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad27ab1-9519-4927-ab2b-7bd25ff21c8e service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:47.094747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad27ab1-9519-4927-ab2b-7bd25ff21c8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:47.102725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad27ab1-9519-4927-ab2b-7bd25ff21c8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:47.104291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad27ab1-9519-4927-ab2b-7bd25ff21c8e service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:47.113823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 719/1425] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:48.017438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90134b37-0c9c-4a0e-8c95-6c77cf4a1a15 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:48.018016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90134b37-0c9c-4a0e-8c95-6c77cf4a1a15 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:48.018016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90134b37-0c9c-4a0e-8c95-6c77cf4a1a15 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:48.019349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:48.024256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfcc70-5c4b-4264-b56a-fc87476b65b7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:48.024665 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfcc70-5c4b-4264-b56a-fc87476b65b7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:48.024821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfcc70-5c4b-4264-b56a-fc87476b65b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:48.042616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f6dfcc70-5c4b-4264-b56a-fc87476b65b7 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:48.043804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f6dfcc70-5c4b-4264-b56a-fc87476b65b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:48.052140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:48.201396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90134b37-0c9c-4a0e-8c95-6c77cf4a1a15 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:48.215256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90134b37-0c9c-4a0e-8c95-6c77cf4a1a15 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:48.220635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185e951-fc92-411e-9fd0-0667b222b5aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:48.221303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185e951-fc92-411e-9fd0-0667b222b5aa service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:48.221588 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185e951-fc92-411e-9fd0-0667b222b5aa service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:48.221862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185e951-fc92-411e-9fd0-0667b222b5aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:48.229760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185e951-fc92-411e-9fd0-0667b222b5aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:48.231325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185e951-fc92-411e-9fd0-0667b222b5aa service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:48.241397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 708/1428] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:49.146704 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116f843-9dd0-4dff-b195-07772e9522fd None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:49.147266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116f843-9dd0-4dff-b195-07772e9522fd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:49.147266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116f843-9dd0-4dff-b195-07772e9522fd None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:49.148611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:49.153557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a8a15-ef63-4b2b-b996-3330f787f027 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:49.153989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a8a15-ef63-4b2b-b996-3330f787f027 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:49.154100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a8a15-ef63-4b2b-b996-3330f787f027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:49.171665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-412a8a15-ef63-4b2b-b996-3330f787f027 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:49.172781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-412a8a15-ef63-4b2b-b996-3330f787f027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:49.180554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:49.329342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116f843-9dd0-4dff-b195-07772e9522fd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:49.343384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116f843-9dd0-4dff-b195-07772e9522fd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:49.348719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c227ba1-5642-4286-b5eb-6e68913046dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:49.349378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227ba1-5642-4286-b5eb-6e68913046dd service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:49.349706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227ba1-5642-4286-b5eb-6e68913046dd service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:49.349938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227ba1-5642-4286-b5eb-6e68913046dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:49.361344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c227ba1-5642-4286-b5eb-6e68913046dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:49.361344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c227ba1-5642-4286-b5eb-6e68913046dd service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:49.370434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 722/1431] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:50.275596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4001f-9c50-4ff4-94b7-60f1b7baee15 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:50.276040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4001f-9c50-4ff4-94b7-60f1b7baee15 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:50.276149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4001f-9c50-4ff4-94b7-60f1b7baee15 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:50.277536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:50.282956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038098-5ff3-4662-bb9d-090030993b5a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:50.283392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038098-5ff3-4662-bb9d-090030993b5a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:50.283514 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69038098-5ff3-4662-bb9d-090030993b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:50.301705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-69038098-5ff3-4662-bb9d-090030993b5a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:50.303086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-69038098-5ff3-4662-bb9d-090030993b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:50.314927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:50.462339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4001f-9c50-4ff4-94b7-60f1b7baee15 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:50.476721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4001f-9c50-4ff4-94b7-60f1b7baee15 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:50.482230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ec7dc-6eec-42ae-9ed4-9ab3331a8c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:50.482853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ec7dc-6eec-42ae-9ed4-9ab3331a8c6f service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:50.483131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ec7dc-6eec-42ae-9ed4-9ab3331a8c6f service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:50.483400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ec7dc-6eec-42ae-9ed4-9ab3331a8c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:50.491298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ec7dc-6eec-42ae-9ed4-9ab3331a8c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:50.492804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ec7dc-6eec-42ae-9ed4-9ab3331a8c6f service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:50.501933 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 711/1434] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:51.420162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dbea1-1f4d-4a3f-bbd6-3fc50a891e54 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:51.420724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dbea1-1f4d-4a3f-bbd6-3fc50a891e54 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:51.420896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dbea1-1f4d-4a3f-bbd6-3fc50a891e54 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:51.422314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:51.427165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7c4b-52db-4932-a8b2-0dbf63904a88 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:51.427592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7c4b-52db-4932-a8b2-0dbf63904a88 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:51.427710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7c4b-52db-4932-a8b2-0dbf63904a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:51.445797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-98ae7c4b-52db-4932-a8b2-0dbf63904a88 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:51.446981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-98ae7c4b-52db-4932-a8b2-0dbf63904a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:51.455293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:51.601988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dbea1-1f4d-4a3f-bbd6-3fc50a891e54 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:51.616153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dbea1-1f4d-4a3f-bbd6-3fc50a891e54 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:51.621438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ec725-a51c-497f-b86a-46ab969b123f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:51.622071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec725-a51c-497f-b86a-46ab969b123f service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:51.622357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec725-a51c-497f-b86a-46ab969b123f service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:51.622761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec725-a51c-497f-b86a-46ab969b123f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:51.631610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ec725-a51c-497f-b86a-46ab969b123f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:51.633268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ec725-a51c-497f-b86a-46ab969b123f service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:51.643458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 725/1437] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:52.539857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667915d-941c-4622-a6c9-4be8d1f91dde None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:52.540438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667915d-941c-4622-a6c9-4be8d1f91dde None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:52.540438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667915d-941c-4622-a6c9-4be8d1f91dde None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:52.541873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:52.547362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2592ca-2db0-45b1-8b34-1e6cf7ce6f33 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:52.547783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2592ca-2db0-45b1-8b34-1e6cf7ce6f33 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:52.547907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2592ca-2db0-45b1-8b34-1e6cf7ce6f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:52.566282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4e2592ca-2db0-45b1-8b34-1e6cf7ce6f33 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:52.567456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4e2592ca-2db0-45b1-8b34-1e6cf7ce6f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:52.575099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:52.721359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667915d-941c-4622-a6c9-4be8d1f91dde None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:52.736270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667915d-941c-4622-a6c9-4be8d1f91dde None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:52.741685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9678dbf-60bd-4dd4-a788-cc3ed34b2880 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:52.742319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678dbf-60bd-4dd4-a788-cc3ed34b2880 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:52.742606 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678dbf-60bd-4dd4-a788-cc3ed34b2880 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:52.742882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9678dbf-60bd-4dd4-a788-cc3ed34b2880 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:52.751705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9678dbf-60bd-4dd4-a788-cc3ed34b2880 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:52.753330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9678dbf-60bd-4dd4-a788-cc3ed34b2880 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:52.763066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 714/1440] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:54.822122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72db448-1d7a-4b67-9eea-a2c4c6a04c25 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:54.822669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72db448-1d7a-4b67-9eea-a2c4c6a04c25 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:54.822669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72db448-1d7a-4b67-9eea-a2c4c6a04c25 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:54.824040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:54.830156 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41cb4-8fd3-4bef-837b-4a281a94e667 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:54.830598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41cb4-8fd3-4bef-837b-4a281a94e667 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:54.830884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a41cb4-8fd3-4bef-837b-4a281a94e667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:54.849036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d1a41cb4-8fd3-4bef-837b-4a281a94e667 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:54.850284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d1a41cb4-8fd3-4bef-837b-4a281a94e667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:54.858257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:28:54.896140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72db448-1d7a-4b67-9eea-a2c4c6a04c25 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:54.908347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72db448-1d7a-4b67-9eea-a2c4c6a04c25 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:54.912130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fac55-2ba0-4e0c-ab3b-aabf28f66717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:54.912868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fac55-2ba0-4e0c-ab3b-aabf28f66717 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:54.913186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fac55-2ba0-4e0c-ab3b-aabf28f66717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:54.913493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fac55-2ba0-4e0c-ab3b-aabf28f66717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:54.922817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fac55-2ba0-4e0c-ab3b-aabf28f66717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:54.924258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fac55-2ba0-4e0c-ab3b-aabf28f66717 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:54.933595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 728/1443] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:57.328660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6336a9-ef17-44be-94d0-ac3954bfb7c1 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:57.329242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6336a9-ef17-44be-94d0-ac3954bfb7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:57.329242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6336a9-ef17-44be-94d0-ac3954bfb7c1 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:57.330586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:57.335751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1c516-908e-4ae7-93ea-c58c1954ef62 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:57.336151 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1c516-908e-4ae7-93ea-c58c1954ef62 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:57.336314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1c516-908e-4ae7-93ea-c58c1954ef62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:57.354145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d1b1c516-908e-4ae7-93ea-c58c1954ef62 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:28:57.355339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d1b1c516-908e-4ae7-93ea-c58c1954ef62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:28:57.363231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:28:57.400658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6336a9-ef17-44be-94d0-ac3954bfb7c1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:28:57.411181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6336a9-ef17-44be-94d0-ac3954bfb7c1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:28:57.414654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945c3fa-f6cd-4a8a-abc9-d77178147777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:28:57.415273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945c3fa-f6cd-4a8a-abc9-d77178147777 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:57.415554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945c3fa-f6cd-4a8a-abc9-d77178147777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:57.415823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945c3fa-f6cd-4a8a-abc9-d77178147777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:57.424491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945c3fa-f6cd-4a8a-abc9-d77178147777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:28:57.425945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945c3fa-f6cd-4a8a-abc9-d77178147777 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:28:57.434982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 717/1446] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:28:59.977400 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ec2bf-b38a-4d17-b52c-3d4a5f81ecd8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:59.977943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ec2bf-b38a-4d17-b52c-3d4a5f81ecd8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:59.977943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ec2bf-b38a-4d17-b52c-3d4a5f81ecd8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:28:59.979356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:28:59.984823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8429a-ab7b-48bb-b117-dde1e9daa225 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:28:59.985243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8429a-ab7b-48bb-b117-dde1e9daa225 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:28:59.985420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8429a-ab7b-48bb-b117-dde1e9daa225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:00.003880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8da8429a-ab7b-48bb-b117-dde1e9daa225 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:00.005148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8da8429a-ab7b-48bb-b117-dde1e9daa225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:00.013193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:00.054771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ec2bf-b38a-4d17-b52c-3d4a5f81ecd8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:00.069457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ec2bf-b38a-4d17-b52c-3d4a5f81ecd8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:00.072938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa3631-91b0-4288-aeb5-53406444e2d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:00.073603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa3631-91b0-4288-aeb5-53406444e2d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:00.073884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa3631-91b0-4288-aeb5-53406444e2d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:00.074165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa3631-91b0-4288-aeb5-53406444e2d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:00.083528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa3631-91b0-4288-aeb5-53406444e2d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:00.084946 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa3631-91b0-4288-aeb5-53406444e2d4 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:00.094712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 731/1449] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:02.408758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c053c8-a711-46ad-8759-293cb4ce9e21 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:02.409369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c053c8-a711-46ad-8759-293cb4ce9e21 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:02.409369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c053c8-a711-46ad-8759-293cb4ce9e21 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:02.410716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:29:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:02.416024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d5427-5445-4ef0-90fb-12dc1b3aec21 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:02.416428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d5427-5445-4ef0-90fb-12dc1b3aec21 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:02.416587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d5427-5445-4ef0-90fb-12dc1b3aec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:02.435529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-004d5427-5445-4ef0-90fb-12dc1b3aec21 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:02.436748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-004d5427-5445-4ef0-90fb-12dc1b3aec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:02.445198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:02.483542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c053c8-a711-46ad-8759-293cb4ce9e21 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:02.485756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f5c053c8-a711-46ad-8759-293cb4ce9e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:02.498496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c053c8-a711-46ad-8759-293cb4ce9e21 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:02.502695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346c3c4-f41b-4423-861f-96f2dce1d1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:02.503328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c3c4-f41b-4423-861f-96f2dce1d1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:02.503615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c3c4-f41b-4423-861f-96f2dce1d1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:02.503891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c3c4-f41b-4423-861f-96f2dce1d1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:02.512850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346c3c4-f41b-4423-861f-96f2dce1d1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:02.514339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346c3c4-f41b-4423-861f-96f2dce1d1ee service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:02.523660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 720/1452] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:04.815895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce0961-824d-4b1c-9d0f-71fe66b07c38 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:04.816482 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce0961-824d-4b1c-9d0f-71fe66b07c38 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:04.816681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce0961-824d-4b1c-9d0f-71fe66b07c38 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:04.817974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:04.823148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bb25b-6b8e-4c9c-9f77-baeae92467dd None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:04.823543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bb25b-6b8e-4c9c-9f77-baeae92467dd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:04.823709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3bb25b-6b8e-4c9c-9f77-baeae92467dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:04.841421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6c3bb25b-6b8e-4c9c-9f77-baeae92467dd None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:04.842593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6c3bb25b-6b8e-4c9c-9f77-baeae92467dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:04.850238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:04.887341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce0961-824d-4b1c-9d0f-71fe66b07c38 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:04.898713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce0961-824d-4b1c-9d0f-71fe66b07c38 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:04.902235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7e6ba-645e-4cbc-b101-082708b2c2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:04.902866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7e6ba-645e-4cbc-b101-082708b2c2de service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:04.903146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7e6ba-645e-4cbc-b101-082708b2c2de service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:04.903419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7e6ba-645e-4cbc-b101-082708b2c2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:04.913084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7e6ba-645e-4cbc-b101-082708b2c2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:04.914547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7e6ba-645e-4cbc-b101-082708b2c2de service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:04.923926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 734/1455] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:07.176038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e582-b30e-4c82-be7e-492d8ab14ca2 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:07.176563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e582-b30e-4c82-be7e-492d8ab14ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:07.176563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e582-b30e-4c82-be7e-492d8ab14ca2 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:07.178055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:07.183365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c12b-f6c2-4363-ada0-5ce90fc6a92b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:07.185737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c12b-f6c2-4363-ada0-5ce90fc6a92b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:07.185737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c12b-f6c2-4363-ada0-5ce90fc6a92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:07.201943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eba7c12b-f6c2-4363-ada0-5ce90fc6a92b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:07.203751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eba7c12b-f6c2-4363-ada0-5ce90fc6a92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:07.211490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:07.250017 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0e582-b30e-4c82-be7e-492d8ab14ca2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:07.260465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0e582-b30e-4c82-be7e-492d8ab14ca2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:07.263960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52cb5b-1e05-45e2-9b35-c860cccb1003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:07.264578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52cb5b-1e05-45e2-9b35-c860cccb1003 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:07.264859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52cb5b-1e05-45e2-9b35-c860cccb1003 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:07.265801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52cb5b-1e05-45e2-9b35-c860cccb1003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:07.274464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52cb5b-1e05-45e2-9b35-c860cccb1003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:07.275887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52cb5b-1e05-45e2-9b35-c860cccb1003 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:07.284930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 723/1458] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:09.511312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba591-3312-4cf1-83d2-bf1c31b07b9f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:09.511884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba591-3312-4cf1-83d2-bf1c31b07b9f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:09.511884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ba591-3312-4cf1-83d2-bf1c31b07b9f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:09.513359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:09.521167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44d29e-4777-4798-8813-6bdaf0620a5f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:09.521609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44d29e-4777-4798-8813-6bdaf0620a5f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:09.521828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44d29e-4777-4798-8813-6bdaf0620a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:09.539759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0b44d29e-4777-4798-8813-6bdaf0620a5f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:09.540937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0b44d29e-4777-4798-8813-6bdaf0620a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:09.549214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:09.585688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ba591-3312-4cf1-83d2-bf1c31b07b9f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:09.597333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ba591-3312-4cf1-83d2-bf1c31b07b9f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:09.600798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74547b7e-d91d-4a23-b3cd-5adb277a3e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:09.601463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b7e-d91d-4a23-b3cd-5adb277a3e8e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:09.601745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b7e-d91d-4a23-b3cd-5adb277a3e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:09.602019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547b7e-d91d-4a23-b3cd-5adb277a3e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:09.611776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74547b7e-d91d-4a23-b3cd-5adb277a3e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:09.613221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74547b7e-d91d-4a23-b3cd-5adb277a3e8e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:09.622341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 737/1461] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:11.875873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e1de2-ab9c-4e3e-a54e-474cfdb74ff8 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:11.876381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e1de2-ab9c-4e3e-a54e-474cfdb74ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:11.876381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e1de2-ab9c-4e3e-a54e-474cfdb74ff8 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:11.877775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:11.883094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594d17c-50c0-442d-8c64-06d0f5b48368 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:11.883534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594d17c-50c0-442d-8c64-06d0f5b48368 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:11.883668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594d17c-50c0-442d-8c64-06d0f5b48368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:11.889778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=79 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:29:11.901438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6594d17c-50c0-442d-8c64-06d0f5b48368 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:11.902626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6594d17c-50c0-442d-8c64-06d0f5b48368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:11.911225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:11.947418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e1de2-ab9c-4e3e-a54e-474cfdb74ff8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:11.953257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=73 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:29:11.958945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e1de2-ab9c-4e3e-a54e-474cfdb74ff8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:11.965187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a3a76-e8d9-49e8-9e98-28b98dea0267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:11.965187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3a76-e8d9-49e8-9e98-28b98dea0267 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:11.965187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3a76-e8d9-49e8-9e98-28b98dea0267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:11.965187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a3a76-e8d9-49e8-9e98-28b98dea0267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:11.973128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a3a76-e8d9-49e8-9e98-28b98dea0267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:11.974614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a3a76-e8d9-49e8-9e98-28b98dea0267 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:11.984531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 726/1464] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:14.181458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6359-b6c0-47b8-b3e3-99c70e8bfd70 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:14.182012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6359-b6c0-47b8-b3e3-99c70e8bfd70 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:14.182012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6359-b6c0-47b8-b3e3-99c70e8bfd70 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:14.183303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:14.189140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5b86-9c6d-4ef0-93c5-8f0cfc0b8145 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:14.189583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5b86-9c6d-4ef0-93c5-8f0cfc0b8145 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:14.189692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec5b86-9c6d-4ef0-93c5-8f0cfc0b8145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:14.207770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2aec5b86-9c6d-4ef0-93c5-8f0cfc0b8145 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:14.208950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2aec5b86-9c6d-4ef0-93c5-8f0cfc0b8145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:14.216664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:14.253630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d6359-b6c0-47b8-b3e3-99c70e8bfd70 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:14.264252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d6359-b6c0-47b8-b3e3-99c70e8bfd70 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:14.267790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b1b63-5e16-4a95-be99-57367bb00739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:14.268417 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1b63-5e16-4a95-be99-57367bb00739 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:14.268702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1b63-5e16-4a95-be99-57367bb00739 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:14.268978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b1b63-5e16-4a95-be99-57367bb00739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:14.277737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b1b63-5e16-4a95-be99-57367bb00739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:14.279159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b1b63-5e16-4a95-be99-57367bb00739 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:14.288329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 740/1467] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:16.473287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc669ab-2c47-4a34-a026-a1bd26321c65 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:16.473844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc669ab-2c47-4a34-a026-a1bd26321c65 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:16.473844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc669ab-2c47-4a34-a026-a1bd26321c65 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:16.475192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:16.480380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0ea47-fd04-41a1-b717-7daf094b925c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:16.480805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0ea47-fd04-41a1-b717-7daf094b925c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:16.480944 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0ea47-fd04-41a1-b717-7daf094b925c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:16.504398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dad0ea47-fd04-41a1-b717-7daf094b925c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:16.505530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dad0ea47-fd04-41a1-b717-7daf094b925c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:16.513466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:16.549775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc669ab-2c47-4a34-a026-a1bd26321c65 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:16.560566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc669ab-2c47-4a34-a026-a1bd26321c65 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:16.564726 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56833e0-365f-4399-a3b8-778513bd3df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:16.568962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56833e0-365f-4399-a3b8-778513bd3df8 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:16.568962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56833e0-365f-4399-a3b8-778513bd3df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:16.568962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56833e0-365f-4399-a3b8-778513bd3df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:16.574907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56833e0-365f-4399-a3b8-778513bd3df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:16.576388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56833e0-365f-4399-a3b8-778513bd3df8 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:16.585847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 729/1470] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:26.497787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=11 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:29:26.581795 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=14 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:29:49.547792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e675674-e9b7-4432-b991-c44c74f14951 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:49.548356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e675674-e9b7-4432-b991-c44c74f14951 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:49.548356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e675674-e9b7-4432-b991-c44c74f14951 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:49.549720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 742/1471] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:49.555178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bcaf-9061-49cb-8fea-7793c769332d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:49.568495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bcaf-9061-49cb-8fea-7793c769332d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:49.568495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54bcaf-9061-49cb-8fea-7793c769332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:49.575891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9a54bcaf-9061-49cb-8fea-7793c769332d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:49.577751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9a54bcaf-9061-49cb-8fea-7793c769332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:49.585696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 730/1472] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:49.633751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373fbb3-944a-4121-858b-5e7c9e56a8c7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:49.634049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373fbb3-944a-4121-858b-5e7c9e56a8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:49.634336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373fbb3-944a-4121-858b-5e7c9e56a8c7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:49.635707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 743/1473] 10.222.0.41 () {54 vars in 962 bytes} [Sat May 28 03:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:29:49.644769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50785d5-ef2b-4c9e-a918-f20659cbfd8d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:49.645062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50785d5-ef2b-4c9e-a918-f20659cbfd8d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:49.645379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50785d5-ef2b-4c9e-a918-f20659cbfd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:49.663858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e50785d5-ef2b-4c9e-a918-f20659cbfd8d None None] MFA Rules not processed for user `a47589bf9e7c4acba200979e3a91a6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:49.665055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e50785d5-ef2b-4c9e-a918-f20659cbfd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:49.671926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 731/1474] 10.222.0.41 () {58 vars in 1059 bytes} [Sat May 28 03:29:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:49.679418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373fbb3-944a-4121-858b-5e7c9e56a8c7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:49.690205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373fbb3-944a-4121-858b-5e7c9e56a8c7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:49.694690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6228aa-b42b-47b8-b3f6-5d3078876317 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47589bf9e7c4acba200979e3a91a6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:49.695733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6228aa-b42b-47b8-b3f6-5d3078876317 service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:49.696039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6228aa-b42b-47b8-b3f6-5d3078876317 service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:49.696318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6228aa-b42b-47b8-b3f6-5d3078876317 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:49.706397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6228aa-b42b-47b8-b3f6-5d3078876317 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:49.707810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6228aa-b42b-47b8-b3f6-5d3078876317 service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:49.717443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 744/1475] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:52.012276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fe9fa-b833-4d29-83d2-7a9a0d9ed60b None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:52.013160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fe9fa-b833-4d29-83d2-7a9a0d9ed60b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:52.013160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fe9fa-b833-4d29-83d2-7a9a0d9ed60b None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:52.014769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 732/1476] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:52.019979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45a76-30b6-4b1b-b88a-02bbb4dfe591 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:52.020418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45a76-30b6-4b1b-b88a-02bbb4dfe591 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:52.020524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd45a76-30b6-4b1b-b88a-02bbb4dfe591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:52.039524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5bd45a76-30b6-4b1b-b88a-02bbb4dfe591 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:52.041814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5bd45a76-30b6-4b1b-b88a-02bbb4dfe591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:52.048839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 745/1477] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:29:52.072089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fe9fa-b833-4d29-83d2-7a9a0d9ed60b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:52.085225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fe9fa-b833-4d29-83d2-7a9a0d9ed60b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:52.089655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6085-33f6-441c-9a37-64430c606edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:52.090306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6085-33f6-441c-9a37-64430c606edd admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:52.092091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6085-33f6-441c-9a37-64430c606edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:52.092091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6085-33f6-441c-9a37-64430c606edd admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:52.092091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef6085-33f6-441c-9a37-64430c606edd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:52.093382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-64ef6085-33f6-441c-9a37-64430c606edd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:29:52.093382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: warnings.warn(msg) May 28 03:29:52.093382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: May 28 03:29:52.095077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef6085-33f6-441c-9a37-64430c606edd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:52.119264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 733/1478] 10.222.0.41 () {56 vars in 1108 bytes} [Sat May 28 03:29:52 2022] GET /identity/v3/projects => generated 2553 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:29:54.447004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8463e7b-1a14-4d3a-8a9a-344e17f4ce9b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:54.447558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8463e7b-1a14-4d3a-8a9a-344e17f4ce9b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:54.447558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8463e7b-1a14-4d3a-8a9a-344e17f4ce9b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:54.448914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 746/1479] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:54.454110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed29-00ee-4698-a39c-4fed5404c143 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:54.454525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed29-00ee-4698-a39c-4fed5404c143 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:54.454713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84ed29-00ee-4698-a39c-4fed5404c143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:54.472721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ae84ed29-00ee-4698-a39c-4fed5404c143 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:54.473943 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ae84ed29-00ee-4698-a39c-4fed5404c143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:54.481704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 734/1480] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:54.529165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8463e7b-1a14-4d3a-8a9a-344e17f4ce9b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:54.539112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8463e7b-1a14-4d3a-8a9a-344e17f4ce9b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:54.544427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad6be7-8d65-491d-8237-bde6fa2675fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:54.545146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad6be7-8d65-491d-8237-bde6fa2675fd service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:54.545959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad6be7-8d65-491d-8237-bde6fa2675fd service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:54.546247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad6be7-8d65-491d-8237-bde6fa2675fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:54.554311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad6be7-8d65-491d-8237-bde6fa2675fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:54.557912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad6be7-8d65-491d-8237-bde6fa2675fd service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:54.565366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 747/1481] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:56.953926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7756-40ff-4094-b7c2-ec1346d77819 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:56.954493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7756-40ff-4094-b7c2-ec1346d77819 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:56.954493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7756-40ff-4094-b7c2-ec1346d77819 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:56.955833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 735/1482] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:56.960977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223ad2-54a6-4938-8483-262def1a631d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:56.961448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223ad2-54a6-4938-8483-262def1a631d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:56.961580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68223ad2-54a6-4938-8483-262def1a631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:56.979493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-68223ad2-54a6-4938-8483-262def1a631d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:56.980681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-68223ad2-54a6-4938-8483-262def1a631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:56.988780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 748/1483] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:29:57.024805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea7756-40ff-4094-b7c2-ec1346d77819 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:57.036119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea7756-40ff-4094-b7c2-ec1346d77819 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:57.039666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b0d0-195b-44fa-9e73-04d702c64fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:57.040296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b0d0-195b-44fa-9e73-04d702c64fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:57.040583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b0d0-195b-44fa-9e73-04d702c64fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:57.040864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b0d0-195b-44fa-9e73-04d702c64fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:57.049709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b0d0-195b-44fa-9e73-04d702c64fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:57.051163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b0d0-195b-44fa-9e73-04d702c64fe2 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:57.060989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 736/1484] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:29:59.291205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85368-a208-451b-9bfa-134695058b93 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:59.291752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85368-a208-451b-9bfa-134695058b93 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:59.291752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85368-a208-451b-9bfa-134695058b93 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:59.293191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 749/1485] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:29:59.298792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca1819-ffe2-4825-a780-89f0a19009f2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:59.299197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca1819-ffe2-4825-a780-89f0a19009f2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:59.299350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca1819-ffe2-4825-a780-89f0a19009f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:59.317615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bcca1819-ffe2-4825-a780-89f0a19009f2 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:29:59.318819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bcca1819-ffe2-4825-a780-89f0a19009f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:29:59.326656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 737/1486] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:29:59.363480 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d85368-a208-451b-9bfa-134695058b93 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:29:59.374715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d85368-a208-451b-9bfa-134695058b93 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:29:59.378241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d674f-88f7-42d7-8e92-4d058c11a2c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:29:59.378871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d674f-88f7-42d7-8e92-4d058c11a2c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:29:59.379158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d674f-88f7-42d7-8e92-4d058c11a2c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:29:59.379441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d674f-88f7-42d7-8e92-4d058c11a2c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:29:59.388233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d674f-88f7-42d7-8e92-4d058c11a2c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:29:59.389684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d674f-88f7-42d7-8e92-4d058c11a2c3 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:29:59.398985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 750/1487] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:30:01.666970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d375c6-1cb7-4aec-9405-86b95c184b89 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:30:01.667594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d375c6-1cb7-4aec-9405-86b95c184b89 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:30:01.667594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d375c6-1cb7-4aec-9405-86b95c184b89 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:30:01.668922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 738/1488] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:30:01.675973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f6a24-0954-49a4-99a4-e308c5c9ed56 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:30:01.676411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f6a24-0954-49a4-99a4-e308c5c9ed56 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:30:01.676516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f6a24-0954-49a4-99a4-e308c5c9ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:30:01.694431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-593f6a24-0954-49a4-99a4-e308c5c9ed56 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:30:01.695611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-593f6a24-0954-49a4-99a4-e308c5c9ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:30:01.704254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 751/1489] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:30:01.744009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d375c6-1cb7-4aec-9405-86b95c184b89 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:30:01.754600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d375c6-1cb7-4aec-9405-86b95c184b89 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:30:01.758137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e928563-b027-4125-bd7b-9a02085c5b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:30:01.758769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e928563-b027-4125-bd7b-9a02085c5b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:30:01.759053 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e928563-b027-4125-bd7b-9a02085c5b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:30:01.759332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e928563-b027-4125-bd7b-9a02085c5b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:30:01.768616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e928563-b027-4125-bd7b-9a02085c5b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:30:01.770084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e928563-b027-4125-bd7b-9a02085c5b3d service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:30:01.779292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 739/1490] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:30:04.323184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c4e1-d67e-4e5f-adee-0ae6691649df None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:30:04.323742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c4e1-d67e-4e5f-adee-0ae6691649df None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:30:04.323742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833c4e1-d67e-4e5f-adee-0ae6691649df None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:30:04.325327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 752/1491] 10.222.0.41 () {54 vars in 926 bytes} [Sat May 28 03:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:30:04.330676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c349d1-0d86-463f-86de-08134352ff94 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:30:04.331118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c349d1-0d86-463f-86de-08134352ff94 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:30:04.331260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c349d1-0d86-463f-86de-08134352ff94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:30:04.349378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f9c349d1-0d86-463f-86de-08134352ff94 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:30:04.350581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f9c349d1-0d86-463f-86de-08134352ff94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:30:04.359224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 740/1492] 10.222.0.41 () {58 vars in 1023 bytes} [Sat May 28 03:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:30:04.398995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833c4e1-d67e-4e5f-adee-0ae6691649df None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:30:04.410455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833c4e1-d67e-4e5f-adee-0ae6691649df None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:30:04.414476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b79c6f-0e5b-4089-9566-7499e8a4ee2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:30:04.415116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79c6f-0e5b-4089-9566-7499e8a4ee2a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:30:04.415402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79c6f-0e5b-4089-9566-7499e8a4ee2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:30:04.415706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b79c6f-0e5b-4089-9566-7499e8a4ee2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:30:04.425350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b79c6f-0e5b-4089-9566-7499e8a4ee2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:30:04.426782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b79c6f-0e5b-4089-9566-7499e8a4ee2a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:30:04.436222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 753/1493] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:30:14.344083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=33 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:30:14.431902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=33 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:32:02.327351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859873dc-badc-44c1-a979-3ecda8cebd81 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.327950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859873dc-badc-44c1-a979-3ecda8cebd81 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.327950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859873dc-badc-44c1-a979-3ecda8cebd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:02.345650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-859873dc-badc-44c1-a979-3ecda8cebd81 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:32:02.346819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-859873dc-badc-44c1-a979-3ecda8cebd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:32:02.354448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 741/1494] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:32:02.364274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88027b35-63c7-4978-acf4-c6a798326bfb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.364752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88027b35-63c7-4978-acf4-c6a798326bfb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.364869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88027b35-63c7-4978-acf4-c6a798326bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:02.383093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-88027b35-63c7-4978-acf4-c6a798326bfb None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:32:02.384274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-88027b35-63c7-4978-acf4-c6a798326bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:32:02.391966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 754/1495] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:32:02.396399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859873dc-badc-44c1-a979-3ecda8cebd81 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:02.405907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859873dc-badc-44c1-a979-3ecda8cebd81 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:02.410441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:02.411081 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.411367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.411659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:02.412361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:32:02.412668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:02.413724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:32:02.413724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: warnings.warn(msg) May 28 03:32:02.413724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: May 28 03:32:02.414604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:02.426339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 742/1496] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:32:02.434098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88027b35-63c7-4978-acf4-c6a798326bfb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:02.443960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88027b35-63c7-4978-acf4-c6a798326bfb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:02.448532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:02.449205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.449495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.449771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:02.450446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:02.452233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:32:02.452233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: warnings.warn(msg) May 28 03:32:02.452233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: May 28 03:32:02.453460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:02.567302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 755/1497] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:32:02 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:32:02.571447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:02.581548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc49c5d-99ee-436b-86a9-f11528d01335 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:02.636854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:02.637552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.637839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.638119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:02.638805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:02.639823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:32:02.639823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: warnings.warn(msg) May 28 03:32:02.639823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: May 28 03:32:02.641014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:02.778300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 743/1498] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:32:02 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:32:02.783121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:02.793348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418db7e-2a2b-4bb4-9d98-caea6d969bc0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:02.798462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:02.799130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.799404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.799684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:02.800406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:02.801447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:32:02.801447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: warnings.warn(msg) May 28 03:32:02.801447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: May 28 03:32:02.802318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:02.814818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 756/1499] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:32:02.818841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:02.828467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edd17d-70e1-4f20-8a10-058220c9d484 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:02.833091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f1c9-9357-48b4-8f4e-f1755262b0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:02.833734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f1c9-9357-48b4-8f4e-f1755262b0fe admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.834025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f1c9-9357-48b4-8f4e-f1755262b0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.834304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f1c9-9357-48b4-8f4e-f1755262b0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:02.835027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f1c9-9357-48b4-8f4e-f1755262b0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:02.836049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-6563f1c9-9357-48b4-8f4e-f1755262b0fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:32:02.836049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: warnings.warn(msg) May 28 03:32:02.836049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: May 28 03:32:02.836924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f1c9-9357-48b4-8f4e-f1755262b0fe admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:02.850504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 744/1500] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:32:02.854511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:02.864903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04848f0-a390-43de-8163-a260656461ae admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:02.869420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2d3f4-f4e5-4a8f-9f1a-1d40c60108ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:02.870281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d3f4-f4e5-4a8f-9f1a-1d40c60108ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.870566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d3f4-f4e5-4a8f-9f1a-1d40c60108ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.870845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d3f4-f4e5-4a8f-9f1a-1d40c60108ab admin admin] PATH_INFO: `/v3/projects/abf172e906cf4cf1b9b99f64d700ea8e/users/34313245a56b40688b3d8e4305dea8b1/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:02.885088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2d3f4-f4e5-4a8f-9f1a-1d40c60108ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf172e906cf4cf1b9b99f64d700ea8e, user_id=34313245a56b40688b3d8e4305dea8b1, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:02.886132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-abb2d3f4-f4e5-4a8f-9f1a-1d40c60108ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:32:02.886132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: warnings.warn(msg) May 28 03:32:02.886132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: May 28 03:32:02.888033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2d3f4-f4e5-4a8f-9f1a-1d40c60108ab admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:02.972208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 757/1501] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:32:02 2022] PUT /identity/v3/projects/abf172e906cf4cf1b9b99f64d700ea8e/users/34313245a56b40688b3d8e4305dea8b1/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:32:02.983465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c352a0e-b71b-41b2-adce-b1d8743cedc3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:02.983756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c352a0e-b71b-41b2-adce-b1d8743cedc3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:02.984293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c352a0e-b71b-41b2-adce-b1d8743cedc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:03.009848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2c352a0e-b71b-41b2-adce-b1d8743cedc3 None None] MFA Rules not processed for user `34313245a56b40688b3d8e4305dea8b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:32:03.013255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2c352a0e-b71b-41b2-adce-b1d8743cedc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:32:03.072776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 745/1502] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:32:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:32:03.086459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501f580-ec21-470e-a799-4eb3bb8a8107 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:03.086705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501f580-ec21-470e-a799-4eb3bb8a8107 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:03.086988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501f580-ec21-470e-a799-4eb3bb8a8107 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:03.088392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 758/1503] 10.222.0.41 () {54 vars in 963 bytes} [Sat May 28 03:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:32:03.093319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df135961-95ea-445e-9288-a3647f6b69d9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:03.093616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df135961-95ea-445e-9288-a3647f6b69d9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:03.093899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df135961-95ea-445e-9288-a3647f6b69d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:03.112544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-df135961-95ea-445e-9288-a3647f6b69d9 None None] MFA Rules not processed for user `7092a719870d4b5eafec8d4c51226b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:32:03.113754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-df135961-95ea-445e-9288-a3647f6b69d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:32:03.155528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:32:03.163169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 746/1504] 10.222.0.41 () {58 vars in 1060 bytes} [Sat May 28 03:32:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:32:03.170787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501f580-ec21-470e-a799-4eb3bb8a8107 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:03.181273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501f580-ec21-470e-a799-4eb3bb8a8107 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:03.184851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca63eae-683e-4a8b-8831-644e24c6d6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:03.187835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca63eae-683e-4a8b-8831-644e24c6d6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:03.187835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca63eae-683e-4a8b-8831-644e24c6d6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:03.187835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca63eae-683e-4a8b-8831-644e24c6d6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:03.194618 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca63eae-683e-4a8b-8831-644e24c6d6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:03.196048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca63eae-683e-4a8b-8831-644e24c6d6c2 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:03.205445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 759/1505] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:32:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:32:03.247402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea04292-d648-41ce-9383-af6f310dbb0d None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:03.247695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea04292-d648-41ce-9383-af6f310dbb0d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:03.247978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea04292-d648-41ce-9383-af6f310dbb0d None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:03.249318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 747/1506] 10.222.0.41 () {52 vars in 829 bytes} [Sat May 28 03:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:32:03.253535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca63eae-683e-4a8b-8831-644e24c6d6c2 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:03.314680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca63eae-683e-4a8b-8831-644e24c6d6c2 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:03.319202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:03.320070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:03.320354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:03.320629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] PATH_INFO: `/v3/users/34313245a56b40688b3d8e4305dea8b1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:03.321781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34313245a56b40688b3d8e4305dea8b1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:03.322792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:32:03.322792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: warnings.warn(msg) May 28 03:32:03.322792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: May 28 03:32:03.323988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:03.578542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34313245a56b40688b3d8e4305dea8b1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:03.591808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34313245a56b40688b3d8e4305dea8b1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:03.610769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34313245a56b40688b3d8e4305dea8b1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:03.611571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34313245a56b40688b3d8e4305dea8b1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:03.615538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:32:03.697807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34313245a56b40688b3d8e4305dea8b1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:03.703916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 760/1507] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:32:03 2022] DELETE /identity/v3/users/34313245a56b40688b3d8e4305dea8b1 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:32:03.708383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518907c1-fe5e-4e74-aa7c-c1d5e83aec09 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:03.708861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518907c1-fe5e-4e74-aa7c-c1d5e83aec09 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:03.708987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518907c1-fe5e-4e74-aa7c-c1d5e83aec09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:03.726150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-518907c1-fe5e-4e74-aa7c-c1d5e83aec09 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:32:03.727433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-518907c1-fe5e-4e74-aa7c-c1d5e83aec09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:32:03.785286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 748/1508] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:32:03.794633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:03.796040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:32:03.854270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f352a4c-8e78-4705-b944-b114c50d3712 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:03.857749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c323c32-70a0-4e64-abfb-852d77027637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:03.858386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323c32-70a0-4e64-abfb-852d77027637 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:03.858668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323c32-70a0-4e64-abfb-852d77027637 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:03.859210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323c32-70a0-4e64-abfb-852d77027637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:03.868535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c323c32-70a0-4e64-abfb-852d77027637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:03.869994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c323c32-70a0-4e64-abfb-852d77027637 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:03.879756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 761/1509] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:32:04.699175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518907c1-fe5e-4e74-aa7c-c1d5e83aec09 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:32:04.700692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-518907c1-fe5e-4e74-aa7c-c1d5e83aec09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:32:04.713551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518907c1-fe5e-4e74-aa7c-c1d5e83aec09 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:32:04.721094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:32:04.723256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:32:04.723256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:32:04.723256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] PATH_INFO: `/v3/projects/abf172e906cf4cf1b9b99f64d700ea8e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:32:04.723816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] RBAC: Authorizing `identity:delete_project(project_id=abf172e906cf4cf1b9b99f64d700ea8e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:32:04.724767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:32:04.724767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: warnings.warn(msg) May 28 03:32:04.724767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: May 28 03:32:04.726874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:32:04.844825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abf172e906cf4cf1b9b99f64d700ea8e', 'request_id': 'req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:04.857907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abf172e906cf4cf1b9b99f64d700ea8e', 'request_id': 'req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:04.864488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abf172e906cf4cf1b9b99f64d700ea8e', 'request_id': 'req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:04.961004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] The token cache is being invalidate because project abf172e906cf4cf1b9b99f64d700ea8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:32:04.961280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:04.961972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b1a77304-32fa-41b8-b5e8-4a20b8e8952f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:32:04.963625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 749/1510] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:32:04 2022] DELETE /identity/v3/projects/abf172e906cf4cf1b9b99f64d700ea8e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:32:13.875124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:32:14.957416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:33:44.170948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0fb56-898e-4444-9bc1-0b8a6902edff None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:44.171508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0fb56-898e-4444-9bc1-0b8a6902edff None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:44.171508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0fb56-898e-4444-9bc1-0b8a6902edff None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:44.172814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:33:44.178158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9eecb-26d2-44b1-87c9-091d1bdb4ced None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:44.178604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9eecb-26d2-44b1-87c9-091d1bdb4ced None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:44.178710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9eecb-26d2-44b1-87c9-091d1bdb4ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:44.196612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4dd9eecb-26d2-44b1-87c9-091d1bdb4ced None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:33:44.197842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4dd9eecb-26d2-44b1-87c9-091d1bdb4ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:33:44.257492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:33:44.426996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0fb56-898e-4444-9bc1-0b8a6902edff None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:33:44.428724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-70a0fb56-898e-4444-9bc1-0b8a6902edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:33:44.503245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0fb56-898e-4444-9bc1-0b8a6902edff None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:33:44.508557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cec06-ed9b-4437-ac3a-871756aa8f9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:33:44.509224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cec06-ed9b-4437-ac3a-871756aa8f9d service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:44.509514 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cec06-ed9b-4437-ac3a-871756aa8f9d service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:44.509794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cec06-ed9b-4437-ac3a-871756aa8f9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:44.517836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cec06-ed9b-4437-ac3a-871756aa8f9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:33:44.519925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cec06-ed9b-4437-ac3a-871756aa8f9d service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:33:44.529228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 763/1513] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:33:45.405084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e78fa-f06a-41a0-8957-3af1825abfdb None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:45.405646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e78fa-f06a-41a0-8957-3af1825abfdb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:45.405764 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e78fa-f06a-41a0-8957-3af1825abfdb None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:45.407098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:33:45.412683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb962ce-b82c-4112-bc76-e32b70fea10b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:45.413145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb962ce-b82c-4112-bc76-e32b70fea10b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:45.413306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb962ce-b82c-4112-bc76-e32b70fea10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:45.431115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ebb962ce-b82c-4112-bc76-e32b70fea10b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:33:45.432274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ebb962ce-b82c-4112-bc76-e32b70fea10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:33:45.441349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:33:45.533537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d485fd-ea27-47e2-a709-5241dfbbb662 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:45.533811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d485fd-ea27-47e2-a709-5241dfbbb662 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:45.534095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d485fd-ea27-47e2-a709-5241dfbbb662 None None] PATH_INFO: `/v3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:45.535952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 752/1516] 10.222.0.27 () {54 vars in 925 bytes} [Sat May 28 03:33:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:33:45.556509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef10d8-8e87-4182-8395-91e37e577cf4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:45.556807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef10d8-8e87-4182-8395-91e37e577cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:45.557094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef10d8-8e87-4182-8395-91e37e577cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:45.575121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-39ef10d8-8e87-4182-8395-91e37e577cf4 None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:33:45.576316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-39ef10d8-8e87-4182-8395-91e37e577cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:33:45.594548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d485fd-ea27-47e2-a709-5241dfbbb662 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:33:45.609638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d485fd-ea27-47e2-a709-5241dfbbb662 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:33:45.615477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fb6c0-e975-4213-932f-519149775b7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:33:45.616116 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fb6c0-e975-4213-932f-519149775b7c service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:45.616400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fb6c0-e975-4213-932f-519149775b7c service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:45.616711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fb6c0-e975-4213-932f-519149775b7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:45.625209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236fb6c0-e975-4213-932f-519149775b7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:33:45.626820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236fb6c0-e975-4213-932f-519149775b7c service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:33:45.636869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 753/1517] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:33:45.640982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 765/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Sat May 28 03:33:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:33:49.738221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498917c9-f396-4b7a-abca-d09e98c41994 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:49.738784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498917c9-f396-4b7a-abca-d09e98c41994 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:49.738784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498917c9-f396-4b7a-abca-d09e98c41994 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:49.740108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 754/1519] 10.222.0.41 () {54 vars in 960 bytes} [Sat May 28 03:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:33:49.745226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36846-4237-4591-be64-1cfe36d98a38 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:49.745652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36846-4237-4591-be64-1cfe36d98a38 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:49.745792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36846-4237-4591-be64-1cfe36d98a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:49.764034 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-11a36846-4237-4591-be64-1cfe36d98a38 None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:33:49.765260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-11a36846-4237-4591-be64-1cfe36d98a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:33:49.773658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 766/1520] 10.222.0.41 () {58 vars in 1057 bytes} [Sat May 28 03:33:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:33:49.781268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498917c9-f396-4b7a-abca-d09e98c41994 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:33:49.802319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498917c9-f396-4b7a-abca-d09e98c41994 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:33:49.808297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b41c7c-7604-449a-85a2-a9e08ce524db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:33:49.808937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b41c7c-7604-449a-85a2-a9e08ce524db service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:49.809251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b41c7c-7604-449a-85a2-a9e08ce524db service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:49.809522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b41c7c-7604-449a-85a2-a9e08ce524db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:49.818909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b41c7c-7604-449a-85a2-a9e08ce524db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:33:49.820463 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b41c7c-7604-449a-85a2-a9e08ce524db service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:33:49.829845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 755/1521] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:33:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:33:56.942112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a7f89-9722-413b-b6d8-3152a69aebf2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:56.942629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a7f89-9722-413b-b6d8-3152a69aebf2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:56.942745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a7f89-9722-413b-b6d8-3152a69aebf2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:56.944024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:33:56.948824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780afec-f5db-40ca-adb0-a7141ee77c7b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:56.949286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780afec-f5db-40ca-adb0-a7141ee77c7b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:56.949415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780afec-f5db-40ca-adb0-a7141ee77c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:56.969358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9780afec-f5db-40ca-adb0-a7141ee77c7b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:33:56.970902 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9780afec-f5db-40ca-adb0-a7141ee77c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:33:56.979732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:33:57.134941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a7f89-9722-413b-b6d8-3152a69aebf2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:33:57.149363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a7f89-9722-413b-b6d8-3152a69aebf2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:33:57.154833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06490b-bf60-4dde-9b77-2823e14fc98e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:33:57.155451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06490b-bf60-4dde-9b77-2823e14fc98e service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:57.155746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06490b-bf60-4dde-9b77-2823e14fc98e service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:57.156025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06490b-bf60-4dde-9b77-2823e14fc98e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:57.164774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06490b-bf60-4dde-9b77-2823e14fc98e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:33:57.166348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06490b-bf60-4dde-9b77-2823e14fc98e service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:33:57.176328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 768/1524] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:33:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:33:58.053041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc433b5-8ea7-4e52-9b3d-1ad675864efc None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:58.053596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc433b5-8ea7-4e52-9b3d-1ad675864efc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:58.053721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc433b5-8ea7-4e52-9b3d-1ad675864efc None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:58.055029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 03:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:33:58.059860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2831e-0c63-4aa5-8fe4-5dc1d4828e9a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:58.060273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2831e-0c63-4aa5-8fe4-5dc1d4828e9a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:58.060426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2831e-0c63-4aa5-8fe4-5dc1d4828e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:58.079065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-70d2831e-0c63-4aa5-8fe4-5dc1d4828e9a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:33:58.080264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-70d2831e-0c63-4aa5-8fe4-5dc1d4828e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:33:58.088121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 03:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:33:58.239324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc433b5-8ea7-4e52-9b3d-1ad675864efc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:33:58.253971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc433b5-8ea7-4e52-9b3d-1ad675864efc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:33:58.259289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eedb2-310c-462a-a1fd-c67f688e38c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:33:58.259916 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eedb2-310c-462a-a1fd-c67f688e38c2 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:33:58.260196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eedb2-310c-462a-a1fd-c67f688e38c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:33:58.260472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eedb2-310c-462a-a1fd-c67f688e38c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:33:58.268573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eedb2-310c-462a-a1fd-c67f688e38c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:33:58.270233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202eedb2-310c-462a-a1fd-c67f688e38c2 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:33:58.279511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 758/1527] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:34:02.322449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740746f4-f12a-424e-be3b-c122e11a4397 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:02.323024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740746f4-f12a-424e-be3b-c122e11a4397 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:02.323024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740746f4-f12a-424e-be3b-c122e11a4397 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:02.324351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:34:02.329399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651cdcb-c417-4675-b466-7b468d92c028 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:02.329811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651cdcb-c417-4675-b466-7b468d92c028 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:02.329965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651cdcb-c417-4675-b466-7b468d92c028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:02.348076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3651cdcb-c417-4675-b466-7b468d92c028 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:34:02.349301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3651cdcb-c417-4675-b466-7b468d92c028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:34:02.356984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:34:02.392994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740746f4-f12a-424e-be3b-c122e11a4397 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:34:02.394812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-740746f4-f12a-424e-be3b-c122e11a4397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:34:02.447701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:34:02.451810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740746f4-f12a-424e-be3b-c122e11a4397 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:34:02.455394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39bc23-c384-4af7-99f6-641a912e0b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:34:02.456036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39bc23-c384-4af7-99f6-641a912e0b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:02.456325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39bc23-c384-4af7-99f6-641a912e0b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:02.456607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39bc23-c384-4af7-99f6-641a912e0b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:02.466033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39bc23-c384-4af7-99f6-641a912e0b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:34:02.467459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39bc23-c384-4af7-99f6-641a912e0b17 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:34:02.476920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 771/1530] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:34:04.753193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae977-a713-49e7-9a11-470968765b79 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:04.753737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae977-a713-49e7-9a11-470968765b79 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:04.753737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dae977-a713-49e7-9a11-470968765b79 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:04.755149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:34:04.760479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bad683-014e-4593-bdc5-da1e02465822 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:04.760893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bad683-014e-4593-bdc5-da1e02465822 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:04.761048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bad683-014e-4593-bdc5-da1e02465822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:04.779280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f6bad683-014e-4593-bdc5-da1e02465822 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:34:04.780470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f6bad683-014e-4593-bdc5-da1e02465822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:34:04.788828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:34:04.823806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae977-a713-49e7-9a11-470968765b79 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:34:04.834845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dae977-a713-49e7-9a11-470968765b79 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:34:04.838658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05924a4b-94a2-4ae6-8111-d5cd84987df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:34:04.839320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05924a4b-94a2-4ae6-8111-d5cd84987df8 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:04.839605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05924a4b-94a2-4ae6-8111-d5cd84987df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:04.839882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05924a4b-94a2-4ae6-8111-d5cd84987df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:04.850069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05924a4b-94a2-4ae6-8111-d5cd84987df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:34:04.851529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05924a4b-94a2-4ae6-8111-d5cd84987df8 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:34:04.863397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 761/1533] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:34:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:34:08.535730 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8efaf3-d57b-4d55-8dd6-2df0d4aff14a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:08.536845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8efaf3-d57b-4d55-8dd6-2df0d4aff14a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:08.537132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8efaf3-d57b-4d55-8dd6-2df0d4aff14a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:08.538721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:34:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:34:08.544763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2f09c-085a-4301-aa0b-490ede0f813c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:08.545238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2f09c-085a-4301-aa0b-490ede0f813c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:08.545392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b2f09c-085a-4301-aa0b-490ede0f813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:08.551676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=58 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:34:08.563342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b7b2f09c-085a-4301-aa0b-490ede0f813c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:34:08.566810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b7b2f09c-085a-4301-aa0b-490ede0f813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:34:08.572426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:34:08.842926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8efaf3-d57b-4d55-8dd6-2df0d4aff14a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:34:08.844643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5a8efaf3-d57b-4d55-8dd6-2df0d4aff14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:34:08.900853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8efaf3-d57b-4d55-8dd6-2df0d4aff14a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:34:08.904430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cea5c-552b-4c06-9648-8aa07c7f8c32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:34:08.905165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cea5c-552b-4c06-9648-8aa07c7f8c32 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:08.905380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cea5c-552b-4c06-9648-8aa07c7f8c32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:08.905658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cea5c-552b-4c06-9648-8aa07c7f8c32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:08.914831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31cea5c-552b-4c06-9648-8aa07c7f8c32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:34:08.916246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31cea5c-552b-4c06-9648-8aa07c7f8c32 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:34:08.925444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 774/1536] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:34:11.123868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75aed3-a37c-49b4-9838-2bb4ddb2b82e None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:11.124383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75aed3-a37c-49b4-9838-2bb4ddb2b82e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:11.124383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75aed3-a37c-49b4-9838-2bb4ddb2b82e None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:11.125744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 03:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:34:11.131543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eedec-0d59-4662-8030-fbe67b735aa5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:11.132001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eedec-0d59-4662-8030-fbe67b735aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:11.132110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eedec-0d59-4662-8030-fbe67b735aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:11.150481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-626eedec-0d59-4662-8030-fbe67b735aa5 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:34:11.151674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-626eedec-0d59-4662-8030-fbe67b735aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:34:11.159380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 03:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:34:11.430093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75aed3-a37c-49b4-9838-2bb4ddb2b82e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:34:11.431758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7a75aed3-a37c-49b4-9838-2bb4ddb2b82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:34:11.445044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75aed3-a37c-49b4-9838-2bb4ddb2b82e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:34:11.458245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275ddef-1588-4bb5-a323-7dfc90b358fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:34:11.458245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ddef-1588-4bb5-a323-7dfc90b358fa service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:34:11.458245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ddef-1588-4bb5-a323-7dfc90b358fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:34:11.458245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275ddef-1588-4bb5-a323-7dfc90b358fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:34:11.458849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275ddef-1588-4bb5-a323-7dfc90b358fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:34:11.460271 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275ddef-1588-4bb5-a323-7dfc90b358fa service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:34:11.469547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 764/1539] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:34:21.144637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=36 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:34:21.465439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=8 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:40.963995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe96e2-4fcc-446f-9eea-158f32dd3bf9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:40.965071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe96e2-4fcc-446f-9eea-158f32dd3bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:40.965600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe96e2-4fcc-446f-9eea-158f32dd3bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:40.999044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d5fe96e2-4fcc-446f-9eea-158f32dd3bf9 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.000606 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d5fe96e2-4fcc-446f-9eea-158f32dd3bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.011860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 776/1540] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.022686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb0dfa-f478-46f4-a89f-aead9885fc7f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.023681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb0dfa-f478-46f4-a89f-aead9885fc7f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.024192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb0dfa-f478-46f4-a89f-aead9885fc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.050106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c2cb0dfa-f478-46f4-a89f-aead9885fc7f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.051756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c2cb0dfa-f478-46f4-a89f-aead9885fc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.060169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 765/1541] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.065232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe96e2-4fcc-446f-9eea-158f32dd3bf9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:41.076631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe96e2-4fcc-446f-9eea-158f32dd3bf9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:41.085836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:41.086769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.087279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.087769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.088722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:41.089273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:41.090517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 03:35:41.090517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: warnings.warn(msg) May 28 03:35:41.090517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: May 28 03:35:41.091801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:41.109569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 777/1542] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:41.117904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb0dfa-f478-46f4-a89f-aead9885fc7f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:41.127691 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb0dfa-f478-46f4-a89f-aead9885fc7f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:41.137397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:41.138346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.138859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.139350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.140259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:41.141537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:35:41.141537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: warnings.warn(msg) May 28 03:35:41.141537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: May 28 03:35:41.143136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:41.282371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 766/1543] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:41.286228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:41.296899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723c22f-04c0-4a8c-a76b-a17082f5ac29 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:41.379737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:41.380923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.381501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.382041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.383020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:41.384410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:35:41.384410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: warnings.warn(msg) May 28 03:35:41.384410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: May 28 03:35:41.386180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:41.487610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 778/1544] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:35:41.492059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:41.506654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81158b-5d02-43ec-b229-cba09009f482 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:41.513346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c790f3-7176-4c31-b94b-10e3124be93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:41.514432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c790f3-7176-4c31-b94b-10e3124be93a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.514918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c790f3-7176-4c31-b94b-10e3124be93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.515369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c790f3-7176-4c31-b94b-10e3124be93a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.516292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c790f3-7176-4c31-b94b-10e3124be93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:41.518142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c790f3-7176-4c31-b94b-10e3124be93a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:41.537568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 767/1545] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:41.542169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:41.557521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e84499-2a0a-4204-8bcb-77c0087fdf57 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:41.562792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0377cf9-9abe-4da3-a5a3-fe6c9f9226c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:41.563857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377cf9-9abe-4da3-a5a3-fe6c9f9226c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.564376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377cf9-9abe-4da3-a5a3-fe6c9f9226c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.564872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0377cf9-9abe-4da3-a5a3-fe6c9f9226c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.565848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0377cf9-9abe-4da3-a5a3-fe6c9f9226c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:41.567733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0377cf9-9abe-4da3-a5a3-fe6c9f9226c7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:41.584366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 779/1546] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:41.588687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c790f3-7176-4c31-b94b-10e3124be93a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:41.604343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c790f3-7176-4c31-b94b-10e3124be93a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:41.609378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79604e-4171-41d3-be55-c60f85fece81 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.609936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79604e-4171-41d3-be55-c60f85fece81 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.610445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79604e-4171-41d3-be55-c60f85fece81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.612976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48a536-b7dc-4c20-916c-7d8938328cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:41.613929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a536-b7dc-4c20-916c-7d8938328cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.614215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a536-b7dc-4c20-916c-7d8938328cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.614495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48a536-b7dc-4c20-916c-7d8938328cb3 admin admin] PATH_INFO: `/v3/projects/3611c72fb0b8472da98ba9b0c0d86843/users/e0eb0ab5585747d098bbb0b18770f4c5/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.636436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48a536-b7dc-4c20-916c-7d8938328cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3611c72fb0b8472da98ba9b0c0d86843, user_id=e0eb0ab5585747d098bbb0b18770f4c5, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:41.637532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-ba48a536-b7dc-4c20-916c-7d8938328cb3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:35:41.637532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: warnings.warn(msg) May 28 03:35:41.637532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: May 28 03:35:41.638639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-da79604e-4171-41d3-be55-c60f85fece81 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.639594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48a536-b7dc-4c20-916c-7d8938328cb3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:41.640328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-da79604e-4171-41d3-be55-c60f85fece81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.659849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 780/1547] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:35:41.661736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836083e-8ce7-4222-903d-3f57db348eb4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.662026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836083e-8ce7-4222-903d-3f57db348eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.662307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836083e-8ce7-4222-903d-3f57db348eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.686468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8836083e-8ce7-4222-903d-3f57db348eb4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.695660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8836083e-8ce7-4222-903d-3f57db348eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.695791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 781/1548] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.697678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849caeb-bdf6-494e-a5e2-340e37a65140 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.697966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849caeb-bdf6-494e-a5e2-340e37a65140 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.698250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849caeb-bdf6-494e-a5e2-340e37a65140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.718349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4849caeb-bdf6-494e-a5e2-340e37a65140 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.721585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4849caeb-bdf6-494e-a5e2-340e37a65140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.728169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 782/1549] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.729437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b1476-ea21-4930-b6b8-42e80947b235 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.729726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b1476-ea21-4930-b6b8-42e80947b235 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.730002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b1476-ea21-4930-b6b8-42e80947b235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.748125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 768/1550] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:41 2022] PUT /identity/v3/projects/3611c72fb0b8472da98ba9b0c0d86843/users/e0eb0ab5585747d098bbb0b18770f4c5/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:41.750241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-907b1476-ea21-4930-b6b8-42e80947b235 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.750241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-907b1476-ea21-4930-b6b8-42e80947b235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.752890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4a31c-faa7-49df-b24a-848509692b63 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.753221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4a31c-faa7-49df-b24a-848509692b63 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.753507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4a31c-faa7-49df-b24a-848509692b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.770665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dcf4a31c-faa7-49df-b24a-848509692b63 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.771829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dcf4a31c-faa7-49df-b24a-848509692b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.809840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 783/1551] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.811029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369bf14-318b-4c1d-bb42-9c52521b2295 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.811314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369bf14-318b-4c1d-bb42-9c52521b2295 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.811590 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369bf14-318b-4c1d-bb42-9c52521b2295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.830903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0369bf14-318b-4c1d-bb42-9c52521b2295 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.832678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0369bf14-318b-4c1d-bb42-9c52521b2295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.834896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 769/1552] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.836150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8826-e5b9-48d8-9271-1fa9400f4f05 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.836442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8826-e5b9-48d8-9271-1fa9400f4f05 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.836719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d8826-e5b9-48d8-9271-1fa9400f4f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.847269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 784/1553] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.848536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5ebce-9695-4a90-a65c-b37cd4724d42 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.848826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5ebce-9695-4a90-a65c-b37cd4724d42 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.849115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5ebce-9695-4a90-a65c-b37cd4724d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.857333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dd0d8826-e5b9-48d8-9271-1fa9400f4f05 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.858518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dd0d8826-e5b9-48d8-9271-1fa9400f4f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.866572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5fa5ebce-9695-4a90-a65c-b37cd4724d42 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.878242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 770/1554] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.878405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5fa5ebce-9695-4a90-a65c-b37cd4724d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.878512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e88fcf-ff25-45ed-afad-027e7040e0c7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.878512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e88fcf-ff25-45ed-afad-027e7040e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.878512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e88fcf-ff25-45ed-afad-027e7040e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.879301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 785/1555] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.879301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f4734-bdb4-4e35-b17a-57867446552d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.879301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f4734-bdb4-4e35-b17a-57867446552d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.879301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f4734-bdb4-4e35-b17a-57867446552d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.897267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d4e88fcf-ff25-45ed-afad-027e7040e0c7 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.899092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d4e88fcf-ff25-45ed-afad-027e7040e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.903387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b15f4734-bdb4-4e35-b17a-57867446552d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.904552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b15f4734-bdb4-4e35-b17a-57867446552d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.913019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 771/1556] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.914323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87324500-0229-4fba-9587-973136067e4a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.914609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87324500-0229-4fba-9587-973136067e4a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.914949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87324500-0229-4fba-9587-973136067e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.916587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 786/1557] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.917367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f4734-bdb4-4e35-b17a-57867446552d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:41.927664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f4734-bdb4-4e35-b17a-57867446552d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:41.932505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:41.933149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-87324500-0229-4fba-9587-973136067e4a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.933279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.933470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.933752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.934337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-87324500-0229-4fba-9587-973136067e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.934474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:41.934783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:41.936408 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:41.942348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 772/1558] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.944645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b298-f2d6-4e8e-8d77-d4ea6f1f8828 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.945004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b298-f2d6-4e8e-8d77-d4ea6f1f8828 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.945471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9b298-f2d6-4e8e-8d77-d4ea6f1f8828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.954697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 787/1559] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:41.954697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:41.961201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb674d-2010-43d5-9f4a-f2d7f909ff96 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:41.965762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c26a92-9c4d-430a-be0f-9825a84996cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:41.966403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26a92-9c4d-430a-be0f-9825a84996cd admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.966689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26a92-9c4d-430a-be0f-9825a84996cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.966965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26a92-9c4d-430a-be0f-9825a84996cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.967679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c26a92-9c4d-430a-be0f-9825a84996cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:41.967983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c26a92-9c4d-430a-be0f-9825a84996cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:41.969602 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c26a92-9c4d-430a-be0f-9825a84996cd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:41.970438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e8a9b298-f2d6-4e8e-8d77-d4ea6f1f8828 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:41.971120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:41.971634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e8a9b298-f2d6-4e8e-8d77-d4ea6f1f8828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:41.979690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 773/1560] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:41.981073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ec1bb-73d2-4326-92db-6cd2d57ce66b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.981407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ec1bb-73d2-4326-92db-6cd2d57ce66b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.984991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ec1bb-73d2-4326-92db-6cd2d57ce66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:41.985125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 788/1561] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:41.985125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfb165-2d8c-4cc9-ba46-fa69028e0cdb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:41.985125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfb165-2d8c-4cc9-ba46-fa69028e0cdb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:41.985125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfb165-2d8c-4cc9-ba46-fa69028e0cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.000662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9b7ec1bb-73d2-4326-92db-6cd2d57ce66b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:42.001895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9b7ec1bb-73d2-4326-92db-6cd2d57ce66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:42.009560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e1bfb165-2d8c-4cc9-ba46-fa69028e0cdb None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:42.013756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 774/1562] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:42.013882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e1bfb165-2d8c-4cc9-ba46-fa69028e0cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:42.013992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ec1bb-73d2-4326-92db-6cd2d57ce66b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.019076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 789/1563] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:42.020905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2cd7-df2e-40ae-87fe-519cec399c54 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.021291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2cd7-df2e-40ae-87fe-519cec399c54 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.021636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa2cd7-df2e-40ae-87fe-519cec399c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.021747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ec1bb-73d2-4326-92db-6cd2d57ce66b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.026485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.027131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.027400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.027684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.029210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:42.029210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.030336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.039414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a9aa2cd7-df2e-40ae-87fe-519cec399c54 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:42.040586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a9aa2cd7-df2e-40ae-87fe-519cec399c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:42.047045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 775/1564] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:42.048493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.050090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 790/1565] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:42.061824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa2cd7-df2e-40ae-87fe-519cec399c54 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.061824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa2cd7-df2e-40ae-87fe-519cec399c54 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.068201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59687c11-d313-4309-b60d-d2d575196640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.068201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687c11-d313-4309-b60d-d2d575196640 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.068201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687c11-d313-4309-b60d-d2d575196640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.069175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59687c11-d313-4309-b60d-d2d575196640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.069175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59687c11-d313-4309-b60d-d2d575196640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.070908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59687c11-d313-4309-b60d-d2d575196640 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.112978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa443e-b6e8-45bc-9026-0cf3aa191716 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.116677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149682fb-ef0f-449f-921e-27844e7ffd95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.119177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149682fb-ef0f-449f-921e-27844e7ffd95 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.119177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149682fb-ef0f-449f-921e-27844e7ffd95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.119177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149682fb-ef0f-449f-921e-27844e7ffd95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.123945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:42.127449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149682fb-ef0f-449f-921e-27844e7ffd95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.129481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149682fb-ef0f-449f-921e-27844e7ffd95 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.139078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 776/1566] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:42.139935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149682fb-ef0f-449f-921e-27844e7ffd95 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.149781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149682fb-ef0f-449f-921e-27844e7ffd95 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.154765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.155429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.155733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.156004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.156705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:42.157011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.158635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.172113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 777/1567] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:42.172909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.180882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 791/1568] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:42.182064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59687c11-d313-4309-b60d-d2d575196640 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.186413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177007da-691a-4474-ad31-45c1382fabe2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.244678 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393d376-bc60-4365-931e-8b23100f171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.245403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393d376-bc60-4365-931e-8b23100f171d admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.245688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393d376-bc60-4365-931e-8b23100f171d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.245972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393d376-bc60-4365-931e-8b23100f171d admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.246663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393d376-bc60-4365-931e-8b23100f171d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.248583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393d376-bc60-4365-931e-8b23100f171d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.249561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59687c11-d313-4309-b60d-d2d575196640 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.254823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.255488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.255767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.256778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.256778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:42.257176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.258966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.273197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 792/1569] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:42.274186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.284417 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45d657-865d-49cf-ab34-f5f272f0e79a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.289015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.289692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.290013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.290248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.290965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:42.291286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.292843 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.305837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 793/1570] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:42.308661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.317718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf573566-fd4d-4571-9496-0788fb5f6264 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.322424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096851b9-4ea4-48a5-b46a-69500b2953e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.323058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096851b9-4ea4-48a5-b46a-69500b2953e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.323338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096851b9-4ea4-48a5-b46a-69500b2953e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.323612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096851b9-4ea4-48a5-b46a-69500b2953e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.324285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096851b9-4ea4-48a5-b46a-69500b2953e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.326232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096851b9-4ea4-48a5-b46a-69500b2953e5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.356410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 778/1571] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:42.357397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393d376-bc60-4365-931e-8b23100f171d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.418519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393d376-bc60-4365-931e-8b23100f171d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.423339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.424186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.424792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.427888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.427888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:42.427888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.428707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.433638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 794/1572] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:42.435100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096851b9-4ea4-48a5-b46a-69500b2953e5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.445074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 779/1573] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:42.446582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.461189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55892a7-5ff5-4975-bfaf-8ce392a111dd admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.521123 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096851b9-4ea4-48a5-b46a-69500b2953e5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.527360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1da916-dfbf-4a66-ad16-07b515523c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.528256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da916-dfbf-4a66-ad16-07b515523c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.528708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da916-dfbf-4a66-ad16-07b515523c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.529236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1da916-dfbf-4a66-ad16-07b515523c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.534349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1da916-dfbf-4a66-ad16-07b515523c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.538790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1da916-dfbf-4a66-ad16-07b515523c3a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.554069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc63ad4-fd02-4373-9cb2-2882291c4724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.554874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc63ad4-fd02-4373-9cb2-2882291c4724 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.555254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc63ad4-fd02-4373-9cb2-2882291c4724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.555606 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc63ad4-fd02-4373-9cb2-2882291c4724 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.556355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc63ad4-fd02-4373-9cb2-2882291c4724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.558387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc63ad4-fd02-4373-9cb2-2882291c4724 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.565924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:42.642991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 795/1574] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:42.642991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1da916-dfbf-4a66-ad16-07b515523c3a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.646784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:42.670997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 780/1575] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:42.672186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc63ad4-fd02-4373-9cb2-2882291c4724 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.684278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc63ad4-fd02-4373-9cb2-2882291c4724 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.713028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1da916-dfbf-4a66-ad16-07b515523c3a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.718104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24568c0-1832-4028-8866-40b18e83c0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.718822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24568c0-1832-4028-8866-40b18e83c0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.719091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24568c0-1832-4028-8866-40b18e83c0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.719375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24568c0-1832-4028-8866-40b18e83c0e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.720057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24568c0-1832-4028-8866-40b18e83c0e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.722053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24568c0-1832-4028-8866-40b18e83c0e0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.746634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e468880-4eda-457b-8908-4c6a8255e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.747357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e468880-4eda-457b-8908-4c6a8255e94a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.747699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e468880-4eda-457b-8908-4c6a8255e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.747975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e468880-4eda-457b-8908-4c6a8255e94a admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.748662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e468880-4eda-457b-8908-4c6a8255e94a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.751330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e468880-4eda-457b-8908-4c6a8255e94a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.824829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 796/1576] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:42.827017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24568c0-1832-4028-8866-40b18e83c0e0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.881976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 781/1577] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:42.882792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e468880-4eda-457b-8908-4c6a8255e94a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:42.888259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24568c0-1832-4028-8866-40b18e83c0e0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.947859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e468880-4eda-457b-8908-4c6a8255e94a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:42.952945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc66fc0-3c3e-4c13-a8dc-63adca1bccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.953869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc66fc0-3c3e-4c13-a8dc-63adca1bccf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.954269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc66fc0-3c3e-4c13-a8dc-63adca1bccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.954616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc66fc0-3c3e-4c13-a8dc-63adca1bccf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.955547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc66fc0-3c3e-4c13-a8dc-63adca1bccf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.958111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73f03b-3db2-471d-b895-d55b6d446d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:42.958259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc66fc0-3c3e-4c13-a8dc-63adca1bccf8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:42.962626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73f03b-3db2-471d-b895-d55b6d446d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:42.962626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73f03b-3db2-471d-b895-d55b6d446d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:42.962626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b73f03b-3db2-471d-b895-d55b6d446d88 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:42.962626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73f03b-3db2-471d-b895-d55b6d446d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:42.962626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b73f03b-3db2-471d-b895-d55b6d446d88 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.057935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 782/1578] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:43.058729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc66fc0-3c3e-4c13-a8dc-63adca1bccf8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.068231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc66fc0-3c3e-4c13-a8dc-63adca1bccf8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.072904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b4c16-d8f7-41c0-90ab-3e473bab5ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.073599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b4c16-d8f7-41c0-90ab-3e473bab5ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.073868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b4c16-d8f7-41c0-90ab-3e473bab5ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.074146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b4c16-d8f7-41c0-90ab-3e473bab5ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.074823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b4c16-d8f7-41c0-90ab-3e473bab5ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.076741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b4c16-d8f7-41c0-90ab-3e473bab5ac4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.098357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 797/1579] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:42 2022] POST /identity/v3/users => generated 292 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:43.100308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73f03b-3db2-471d-b895-d55b6d446d88 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.109093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b73f03b-3db2-471d-b895-d55b6d446d88 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.113988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5a581-b856-4f36-9afe-acd5dd1a2871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.114619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a581-b856-4f36-9afe-acd5dd1a2871 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.114902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a581-b856-4f36-9afe-acd5dd1a2871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.115175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5a581-b856-4f36-9afe-acd5dd1a2871 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.115865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5a581-b856-4f36-9afe-acd5dd1a2871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.117817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5a581-b856-4f36-9afe-acd5dd1a2871 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.208758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 783/1580] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:43 2022] POST /identity/v3/projects => generated 391 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:43.209648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b4c16-d8f7-41c0-90ab-3e473bab5ac4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.236157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 798/1581] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:43 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:43.237847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5a581-b856-4f36-9afe-acd5dd1a2871 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.248396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5a581-b856-4f36-9afe-acd5dd1a2871 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.252067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:43.256294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:43.272914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b4c16-d8f7-41c0-90ab-3e473bab5ac4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.277845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0edd-7fa1-4775-97f9-b38b63c89612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.278538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0edd-7fa1-4775-97f9-b38b63c89612 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.278821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0edd-7fa1-4775-97f9-b38b63c89612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.279096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b0edd-7fa1-4775-97f9-b38b63c89612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.279827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b0edd-7fa1-4775-97f9-b38b63c89612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.281492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b0edd-7fa1-4775-97f9-b38b63c89612 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.292034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 784/1582] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.293010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0edd-7fa1-4775-97f9-b38b63c89612 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.302507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b0edd-7fa1-4775-97f9-b38b63c89612 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.305468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f439-a9d2-4c24-8d7c-8abdc7d05cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.306141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f439-a9d2-4c24-8d7c-8abdc7d05cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.306427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f439-a9d2-4c24-8d7c-8abdc7d05cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.306705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f439-a9d2-4c24-8d7c-8abdc7d05cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.307198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b80540-c8e7-47ec-bb52-78a99db5ca0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.307385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57f439-a9d2-4c24-8d7c-8abdc7d05cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.307899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80540-c8e7-47ec-bb52-78a99db5ca0a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.308187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80540-c8e7-47ec-bb52-78a99db5ca0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.308535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80540-c8e7-47ec-bb52-78a99db5ca0a admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.309240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b80540-c8e7-47ec-bb52-78a99db5ca0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.309350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57f439-a9d2-4c24-8d7c-8abdc7d05cf7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.311176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b80540-c8e7-47ec-bb52-78a99db5ca0a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.423586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 799/1583] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:43 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:43.423586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f439-a9d2-4c24-8d7c-8abdc7d05cf7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.431164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f439-a9d2-4c24-8d7c-8abdc7d05cf7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.435694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab2ac4-e332-42a0-a063-13ab134c0525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.436358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2ac4-e332-42a0-a063-13ab134c0525 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.436639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2ac4-e332-42a0-a063-13ab134c0525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.436912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2ac4-e332-42a0-a063-13ab134c0525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.437651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab2ac4-e332-42a0-a063-13ab134c0525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.439224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab2ac4-e332-42a0-a063-13ab134c0525 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.456022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 785/1584] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:43 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:43.457591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b80540-c8e7-47ec-bb52-78a99db5ca0a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.460254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 800/1585] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.461368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab2ac4-e332-42a0-a063-13ab134c0525 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.468869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b80540-c8e7-47ec-bb52-78a99db5ca0a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.471891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab2ac4-e332-42a0-a063-13ab134c0525 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.476448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5ff9-8a24-4f6d-a9c0-814b7158a459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.476448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5ff9-8a24-4f6d-a9c0-814b7158a459 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.476448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5ff9-8a24-4f6d-a9c0-814b7158a459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.476448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f5ff9-8a24-4f6d-a9c0-814b7158a459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.476849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f5ff9-8a24-4f6d-a9c0-814b7158a459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.476934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adefaf1-9ca2-4699-a455-ba80af6080e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.477381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adefaf1-9ca2-4699-a455-ba80af6080e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.477670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adefaf1-9ca2-4699-a455-ba80af6080e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.477843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f5ff9-8a24-4f6d-a9c0-814b7158a459 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.477976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adefaf1-9ca2-4699-a455-ba80af6080e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.478619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adefaf1-9ca2-4699-a455-ba80af6080e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.480550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adefaf1-9ca2-4699-a455-ba80af6080e1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.489693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 786/1586] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.490403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5ff9-8a24-4f6d-a9c0-814b7158a459 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.499970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f5ff9-8a24-4f6d-a9c0-814b7158a459 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.505257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b1cb-65a4-408b-9206-9b60f2903449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.505974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b1cb-65a4-408b-9206-9b60f2903449 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.506338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b1cb-65a4-408b-9206-9b60f2903449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.506675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92b1cb-65a4-408b-9206-9b60f2903449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.507454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92b1cb-65a4-408b-9206-9b60f2903449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.509164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92b1cb-65a4-408b-9206-9b60f2903449 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.518925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 787/1587] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.520357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b1cb-65a4-408b-9206-9b60f2903449 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.531335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92b1cb-65a4-408b-9206-9b60f2903449 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.538224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfb532-84b2-45cf-86cd-99f29c60553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.539059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfb532-84b2-45cf-86cd-99f29c60553e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.540744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfb532-84b2-45cf-86cd-99f29c60553e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.540744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfb532-84b2-45cf-86cd-99f29c60553e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.540744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfb532-84b2-45cf-86cd-99f29c60553e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.542277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfb532-84b2-45cf-86cd-99f29c60553e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.552544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 788/1588] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.552544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfb532-84b2-45cf-86cd-99f29c60553e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.565168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfb532-84b2-45cf-86cd-99f29c60553e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.571284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d60b4-7048-4aa6-9ad1-b15ef049f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.574740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d60b4-7048-4aa6-9ad1-b15ef049f99f admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.574740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d60b4-7048-4aa6-9ad1-b15ef049f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.574740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d60b4-7048-4aa6-9ad1-b15ef049f99f admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.574740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d60b4-7048-4aa6-9ad1-b15ef049f99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.578070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d60b4-7048-4aa6-9ad1-b15ef049f99f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.585720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 801/1589] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:43 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:43.590323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adefaf1-9ca2-4699-a455-ba80af6080e1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.595446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 789/1590] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.597055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d60b4-7048-4aa6-9ad1-b15ef049f99f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.607241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adefaf1-9ca2-4699-a455-ba80af6080e1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.612254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d60b4-7048-4aa6-9ad1-b15ef049f99f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.614993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cd8a8-f9c7-4c97-82d9-9cef4811972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.614993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cd8a8-f9c7-4c97-82d9-9cef4811972b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.615263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cd8a8-f9c7-4c97-82d9-9cef4811972b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.615356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cd8a8-f9c7-4c97-82d9-9cef4811972b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.616059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cd8a8-f9c7-4c97-82d9-9cef4811972b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.617370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b720e99-40e9-4e9c-94e1-7646b976a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.617702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cd8a8-f9c7-4c97-82d9-9cef4811972b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.618084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b720e99-40e9-4e9c-94e1-7646b976a8ec admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.618369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b720e99-40e9-4e9c-94e1-7646b976a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.618702 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b720e99-40e9-4e9c-94e1-7646b976a8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.619562 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b720e99-40e9-4e9c-94e1-7646b976a8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.621346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b720e99-40e9-4e9c-94e1-7646b976a8ec admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.629672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 802/1591] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.630617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cd8a8-f9c7-4c97-82d9-9cef4811972b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.636487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 790/1592] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.636487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b720e99-40e9-4e9c-94e1-7646b976a8ec admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.641992 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cd8a8-f9c7-4c97-82d9-9cef4811972b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.647623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744c846-8949-48f7-bbe0-87f5a8b39421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.648277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744c846-8949-48f7-bbe0-87f5a8b39421 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.648564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744c846-8949-48f7-bbe0-87f5a8b39421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.648679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b720e99-40e9-4e9c-94e1-7646b976a8ec admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.648859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744c846-8949-48f7-bbe0-87f5a8b39421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.649648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744c846-8949-48f7-bbe0-87f5a8b39421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.651249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744c846-8949-48f7-bbe0-87f5a8b39421 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.655047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfa215-5797-4c69-8820-572a8fa361ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.655047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfa215-5797-4c69-8820-572a8fa361ac admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.655047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfa215-5797-4c69-8820-572a8fa361ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.655358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfa215-5797-4c69-8820-572a8fa361ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.657136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfa215-5797-4c69-8820-572a8fa361ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.658230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfa215-5797-4c69-8820-572a8fa361ac admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.663116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 803/1593] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.664366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744c846-8949-48f7-bbe0-87f5a8b39421 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.669251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 791/1594] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.670139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfa215-5797-4c69-8820-572a8fa361ac admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.684093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfa215-5797-4c69-8820-572a8fa361ac admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.687089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744c846-8949-48f7-bbe0-87f5a8b39421 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.693246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a81cd0-06ab-49ec-b351-0740f3073aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.693403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f56b4e-4d96-4015-90a6-38f92e5b27cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.693976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f56b4e-4d96-4015-90a6-38f92e5b27cd admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.694259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f56b4e-4d96-4015-90a6-38f92e5b27cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.695495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a81cd0-06ab-49ec-b351-0740f3073aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.695653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f56b4e-4d96-4015-90a6-38f92e5b27cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.695746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a81cd0-06ab-49ec-b351-0740f3073aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.695746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a81cd0-06ab-49ec-b351-0740f3073aa8 admin admin] PATH_INFO: `/v3/projects/d9bf41aa6e93483c9c73351cefaf7f38/users/cbea6fa382544157b25628d3446fa3f8/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.695953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f56b4e-4d96-4015-90a6-38f92e5b27cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.697199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f56b4e-4d96-4015-90a6-38f92e5b27cd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.709800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 792/1595] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.710788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f56b4e-4d96-4015-90a6-38f92e5b27cd admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.717064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a81cd0-06ab-49ec-b351-0740f3073aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9bf41aa6e93483c9c73351cefaf7f38, user_id=cbea6fa382544157b25628d3446fa3f8, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.720466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a81cd0-06ab-49ec-b351-0740f3073aa8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.722712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f56b4e-4d96-4015-90a6-38f92e5b27cd admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.728236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671609c3-066b-4655-9699-adf1d2ac843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.728935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671609c3-066b-4655-9699-adf1d2ac843c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.729378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671609c3-066b-4655-9699-adf1d2ac843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.729657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671609c3-066b-4655-9699-adf1d2ac843c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.730379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671609c3-066b-4655-9699-adf1d2ac843c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.732093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671609c3-066b-4655-9699-adf1d2ac843c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.744921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 793/1596] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.747028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671609c3-066b-4655-9699-adf1d2ac843c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.756741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671609c3-066b-4655-9699-adf1d2ac843c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.761590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69675dfd-9550-495e-bd21-0c59f4c3f97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.762255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69675dfd-9550-495e-bd21-0c59f4c3f97d admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.762543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69675dfd-9550-495e-bd21-0c59f4c3f97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.762834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69675dfd-9550-495e-bd21-0c59f4c3f97d admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.763552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69675dfd-9550-495e-bd21-0c59f4c3f97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.766046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69675dfd-9550-495e-bd21-0c59f4c3f97d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.776635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 794/1597] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:43.777536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69675dfd-9550-495e-bd21-0c59f4c3f97d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.787767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69675dfd-9550-495e-bd21-0c59f4c3f97d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.792597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69dc4-2c2d-4556-b8d1-9c7b5350baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.793492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69dc4-2c2d-4556-b8d1-9c7b5350baf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.793764 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69dc4-2c2d-4556-b8d1-9c7b5350baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.794617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca69dc4-2c2d-4556-b8d1-9c7b5350baf9 admin admin] PATH_INFO: `/v3/projects/70eaf602c8db4ec1bfb26e5eb3b13c82/users/dc6a3335562448d1934fa18e8f51b119/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.810071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca69dc4-2c2d-4556-b8d1-9c7b5350baf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70eaf602c8db4ec1bfb26e5eb3b13c82, user_id=dc6a3335562448d1934fa18e8f51b119, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.812738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca69dc4-2c2d-4556-b8d1-9c7b5350baf9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.836422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 804/1598] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:43 2022] PUT /identity/v3/projects/d9bf41aa6e93483c9c73351cefaf7f38/users/cbea6fa382544157b25628d3446fa3f8/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:43.837560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a81cd0-06ab-49ec-b351-0740f3073aa8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.905415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a81cd0-06ab-49ec-b351-0740f3073aa8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.910390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612e217-00c1-40d8-aa4e-8068b18bab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.911368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612e217-00c1-40d8-aa4e-8068b18bab83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.911653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612e217-00c1-40d8-aa4e-8068b18bab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.911930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612e217-00c1-40d8-aa4e-8068b18bab83 admin admin] PATH_INFO: `/v3/projects/68e1a9e256f64e46919b1e362b4f68d8/users/9d420e4b554e4deab21b89094e0547ca/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:43.925280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 795/1599] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:43 2022] PUT /identity/v3/projects/70eaf602c8db4ec1bfb26e5eb3b13c82/users/dc6a3335562448d1934fa18e8f51b119/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:43.926668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69dc4-2c2d-4556-b8d1-9c7b5350baf9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:43.928959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612e217-00c1-40d8-aa4e-8068b18bab83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e1a9e256f64e46919b1e362b4f68d8, user_id=9d420e4b554e4deab21b89094e0547ca, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:43.934696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612e217-00c1-40d8-aa4e-8068b18bab83 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:43.983365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:43.987628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca69dc4-2c2d-4556-b8d1-9c7b5350baf9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:43.992891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f70c0-75c6-4db4-85a8-5c3675a57918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:43.993821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f70c0-75c6-4db4-85a8-5c3675a57918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:43.994103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f70c0-75c6-4db4-85a8-5c3675a57918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:43.994377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f70c0-75c6-4db4-85a8-5c3675a57918 admin admin] PATH_INFO: `/v3/projects/c547fb861a0e4888b6bef6c30c5f0c73/users/4c9b89c578a34597b143cb9294946300/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.009064 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f70c0-75c6-4db4-85a8-5c3675a57918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c547fb861a0e4888b6bef6c30c5f0c73, user_id=4c9b89c578a34597b143cb9294946300, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.011897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f70c0-75c6-4db4-85a8-5c3675a57918 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.073801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 805/1600] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:43 2022] PUT /identity/v3/projects/68e1a9e256f64e46919b1e362b4f68d8/users/9d420e4b554e4deab21b89094e0547ca/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:44.074832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612e217-00c1-40d8-aa4e-8068b18bab83 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.080865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:44.110030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 796/1601] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:43 2022] PUT /identity/v3/projects/c547fb861a0e4888b6bef6c30c5f0c73/users/4c9b89c578a34597b143cb9294946300/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:44.112020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f70c0-75c6-4db4-85a8-5c3675a57918 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.150686 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612e217-00c1-40d8-aa4e-8068b18bab83 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.194614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f70c0-75c6-4db4-85a8-5c3675a57918 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.201186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e67482-35d4-4b32-8597-938de9850e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.202189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e67482-35d4-4b32-8597-938de9850e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.202578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e67482-35d4-4b32-8597-938de9850e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.202958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e67482-35d4-4b32-8597-938de9850e54 admin admin] PATH_INFO: `/v3/projects/fc3ad7416d27411cb8ddbce0a3da18eb/users/46e48b861bc2498c9670de030c783f54/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.214302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a63c0-d6cc-4bbd-9b83-34cd237efae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.215296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a63c0-d6cc-4bbd-9b83-34cd237efae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.215438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a63c0-d6cc-4bbd-9b83-34cd237efae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.215893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a63c0-d6cc-4bbd-9b83-34cd237efae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.216628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a63c0-d6cc-4bbd-9b83-34cd237efae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.218289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a63c0-d6cc-4bbd-9b83-34cd237efae4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.224327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e67482-35d4-4b32-8597-938de9850e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3ad7416d27411cb8ddbce0a3da18eb, user_id=46e48b861bc2498c9670de030c783f54, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.227699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e67482-35d4-4b32-8597-938de9850e54 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.230004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 806/1602] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:44.231070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a63c0-d6cc-4bbd-9b83-34cd237efae4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.241868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a63c0-d6cc-4bbd-9b83-34cd237efae4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.247031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa7427-f3c5-45a3-ae87-6148659d7f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.247896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7427-f3c5-45a3-ae87-6148659d7f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.248201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7427-f3c5-45a3-ae87-6148659d7f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.248452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa7427-f3c5-45a3-ae87-6148659d7f0f admin admin] PATH_INFO: `/v3/projects/59256a4c200e481c8482bf81eee0b4bf/users/9b9ccd945d4b4e72bbd47c7026ef041a/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.262976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa7427-f3c5-45a3-ae87-6148659d7f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=59256a4c200e481c8482bf81eee0b4bf, user_id=9b9ccd945d4b4e72bbd47c7026ef041a, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.265761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa7427-f3c5-45a3-ae87-6148659d7f0f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.342355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 807/1603] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:44 2022] PUT /identity/v3/projects/59256a4c200e481c8482bf81eee0b4bf/users/9b9ccd945d4b4e72bbd47c7026ef041a/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:44.352594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc211-af9e-498b-8b8f-c1d7bfe30525 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.352594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc211-af9e-498b-8b8f-c1d7bfe30525 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.352594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc211-af9e-498b-8b8f-c1d7bfe30525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.373075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 797/1604] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:44 2022] PUT /identity/v3/projects/fc3ad7416d27411cb8ddbce0a3da18eb/users/46e48b861bc2498c9670de030c783f54/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:44.374450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bd82-979f-4840-825c-9a0023c8ec06 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.374740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bd82-979f-4840-825c-9a0023c8ec06 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.375020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073bd82-979f-4840-825c-9a0023c8ec06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.388784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0bdfc211-af9e-498b-8b8f-c1d7bfe30525 None None] MFA Rules not processed for user `cbea6fa382544157b25628d3446fa3f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:44.390031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0bdfc211-af9e-498b-8b8f-c1d7bfe30525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:44.411532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4073bd82-979f-4840-825c-9a0023c8ec06 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:44.412612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4073bd82-979f-4840-825c-9a0023c8ec06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:44.481200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 808/1605] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:44.481200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d5ba8-e215-459b-bff6-8b4c62573781 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.481200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d5ba8-e215-459b-bff6-8b4c62573781 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.481200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d5ba8-e215-459b-bff6-8b4c62573781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.495981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 798/1606] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:44.497393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8020e4c-3c55-4210-9371-8abe628c5b3c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.497682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8020e4c-3c55-4210-9371-8abe628c5b3c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.498022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8020e4c-3c55-4210-9371-8abe628c5b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.506451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3a6d5ba8-e215-459b-bff6-8b4c62573781 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:44.508245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a6d5ba8-e215-459b-bff6-8b4c62573781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:44.516811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 809/1607] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:44.522710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d5ba8-e215-459b-bff6-8b4c62573781 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.532132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a8020e4c-3c55-4210-9371-8abe628c5b3c None None] MFA Rules not processed for user `4c9b89c578a34597b143cb9294946300`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:44.533652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a8020e4c-3c55-4210-9371-8abe628c5b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:44.536119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d5ba8-e215-459b-bff6-8b4c62573781 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.543313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00db3ce-15d5-42b0-b187-fe5398e31ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.544201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db3ce-15d5-42b0-b187-fe5398e31ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.544484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db3ce-15d5-42b0-b187-fe5398e31ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.544791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db3ce-15d5-42b0-b187-fe5398e31ce7 admin admin] PATH_INFO: `/v3/projects/5cab8255c6134aebb9c93493a2342cb1/users/b6709eed72ca46468fd298929ae0f764/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.548483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:44.561995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00db3ce-15d5-42b0-b187-fe5398e31ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cab8255c6134aebb9c93493a2342cb1, user_id=b6709eed72ca46468fd298929ae0f764, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.566025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00db3ce-15d5-42b0-b187-fe5398e31ce7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.601871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:44.619248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 799/1608] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:44.621068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8e707-b667-465e-8052-ab0fc2cbd961 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.621543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8e707-b667-465e-8052-ab0fc2cbd961 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.623564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8e707-b667-465e-8052-ab0fc2cbd961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.634497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 810/1609] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:44 2022] PUT /identity/v3/projects/5cab8255c6134aebb9c93493a2342cb1/users/b6709eed72ca46468fd298929ae0f764/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:44.635876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae98d4-8e88-465d-8820-8f4d454ec160 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.636271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae98d4-8e88-465d-8820-8f4d454ec160 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.636648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae98d4-8e88-465d-8820-8f4d454ec160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.678254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-02ae98d4-8e88-465d-8820-8f4d454ec160 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:44.681196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-02ae98d4-8e88-465d-8820-8f4d454ec160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:44.682487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6da8e707-b667-465e-8052-ab0fc2cbd961 None None] MFA Rules not processed for user `9b9ccd945d4b4e72bbd47c7026ef041a`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:44.683859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6da8e707-b667-465e-8052-ab0fc2cbd961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:44.777268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 811/1610] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:44.778174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae98d4-8e88-465d-8820-8f4d454ec160 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.780000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 800/1611] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:44.784484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8e707-b667-465e-8052-ab0fc2cbd961 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.786680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6da8e707-b667-465e-8052-ab0fc2cbd961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:44.849332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8e707-b667-465e-8052-ab0fc2cbd961 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.854132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4f41-d60a-4a81-8249-13c5335fd81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.854817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4f41-d60a-4a81-8249-13c5335fd81a service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.855138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4f41-d60a-4a81-8249-13c5335fd81a service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.855598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e4f41-d60a-4a81-8249-13c5335fd81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.856525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae98d4-8e88-465d-8820-8f4d454ec160 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.860165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05fb07-f4b6-4a7e-baca-6a60c1bad197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.861401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05fb07-f4b6-4a7e-baca-6a60c1bad197 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.861401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05fb07-f4b6-4a7e-baca-6a60c1bad197 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.861810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05fb07-f4b6-4a7e-baca-6a60c1bad197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.866542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e4f41-d60a-4a81-8249-13c5335fd81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.868182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e4f41-d60a-4a81-8249-13c5335fd81a service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.881568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 801/1612] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:44.882409 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4f41-d60a-4a81-8249-13c5335fd81a service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.891948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e4f41-d60a-4a81-8249-13c5335fd81a service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.896115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e3162-3ae7-4f7f-9618-09798a614279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.902246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3162-3ae7-4f7f-9618-09798a614279 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.902246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3162-3ae7-4f7f-9618-09798a614279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.902246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3162-3ae7-4f7f-9618-09798a614279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.907858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e3162-3ae7-4f7f-9618-09798a614279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.908814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e3162-3ae7-4f7f-9618-09798a614279 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.921358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 802/1613] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:44.923289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321c130-47ce-4c70-99f7-e0e3b8501b37 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.923806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321c130-47ce-4c70-99f7-e0e3b8501b37 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.924314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e321c130-47ce-4c70-99f7-e0e3b8501b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.946332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05fb07-f4b6-4a7e-baca-6a60c1bad197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.948003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05fb07-f4b6-4a7e-baca-6a60c1bad197 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.950666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e321c130-47ce-4c70-99f7-e0e3b8501b37 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:44.952173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e321c130-47ce-4c70-99f7-e0e3b8501b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:44.959459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 812/1614] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:44.961023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c860-a26f-4aad-b03e-b17b97d28237 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.961358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c860-a26f-4aad-b03e-b17b97d28237 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.961645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c860-a26f-4aad-b03e-b17b97d28237 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.965289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 803/1615] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:44.965435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 813/1616] 10.222.0.41 () {54 vars in 961 bytes} [Sat May 28 03:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:35:44.965539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321c130-47ce-4c70-99f7-e0e3b8501b37 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.968475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173c860-a26f-4aad-b03e-b17b97d28237 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:44.979164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e321c130-47ce-4c70-99f7-e0e3b8501b37 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.981467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173c860-a26f-4aad-b03e-b17b97d28237 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:44.984972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9f238-4a84-4fbd-9c1c-6a4716516f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.986278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336082f-78ac-424a-82b4-776dd8414f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:44.986554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9f238-4a84-4fbd-9c1c-6a4716516f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.986950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336082f-78ac-424a-82b4-776dd8414f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:44.987161 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9f238-4a84-4fbd-9c1c-6a4716516f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.987285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336082f-78ac-424a-82b4-776dd8414f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:44.987505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336082f-78ac-424a-82b4-776dd8414f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.987866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9f238-4a84-4fbd-9c1c-6a4716516f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:44.989345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9f238-4a84-4fbd-9c1c-6a4716516f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:44.992003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9f238-4a84-4fbd-9c1c-6a4716516f8a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:44.999392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336082f-78ac-424a-82b4-776dd8414f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:45.001189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336082f-78ac-424a-82b4-776dd8414f98 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:45.016304 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 814/1617] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:45.017772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab80165-33b2-46fb-843a-07b2d6dfdda1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.018072 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab80165-33b2-46fb-843a-07b2d6dfdda1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.018398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab80165-33b2-46fb-843a-07b2d6dfdda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.038390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dab80165-33b2-46fb-843a-07b2d6dfdda1 None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:45.039594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dab80165-33b2-46fb-843a-07b2d6dfdda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:45.066076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:45.098677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 804/1618] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:44 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:45.099824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9f238-4a84-4fbd-9c1c-6a4716516f8a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:45.140430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 815/1619] 10.222.0.41 () {58 vars in 1058 bytes} [Sat May 28 03:35:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:45.151794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab80165-33b2-46fb-843a-07b2d6dfdda1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:45.166947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab80165-33b2-46fb-843a-07b2d6dfdda1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:45.212833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:45.228899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9f238-4a84-4fbd-9c1c-6a4716516f8a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:45.241379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33eec4-aabc-4759-a991-34991b1a851b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:45.241992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33eec4-aabc-4759-a991-34991b1a851b service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.242278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33eec4-aabc-4759-a991-34991b1a851b service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.242557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33eec4-aabc-4759-a991-34991b1a851b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.259755 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b0def-636a-4514-af40-3fe1f98b4d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:45.260840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0def-636a-4514-af40-3fe1f98b4d1e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.261319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0def-636a-4514-af40-3fe1f98b4d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.261973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b0def-636a-4514-af40-3fe1f98b4d1e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.263061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b0def-636a-4514-af40-3fe1f98b4d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:45.265757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b0def-636a-4514-af40-3fe1f98b4d1e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:45.350167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33eec4-aabc-4759-a991-34991b1a851b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:45.352660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33eec4-aabc-4759-a991-34991b1a851b service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:45.367415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 805/1620] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:45.370609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33eec4-aabc-4759-a991-34991b1a851b service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:45.398082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 816/1621] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:45 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:45.402598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b0def-636a-4514-af40-3fe1f98b4d1e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:45.428925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b0def-636a-4514-af40-3fe1f98b4d1e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:45.438699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f9ab4-c246-471a-8f12-fe689c87a8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:45.440378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f9ab4-c246-471a-8f12-fe689c87a8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.440671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f9ab4-c246-471a-8f12-fe689c87a8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.440957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f9ab4-c246-471a-8f12-fe689c87a8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.441821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f9ab4-c246-471a-8f12-fe689c87a8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:45.444815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f9ab4-c246-471a-8f12-fe689c87a8bf admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:45.480940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 817/1622] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:45.482124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f9ab4-c246-471a-8f12-fe689c87a8bf admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:45.500866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f9ab4-c246-471a-8f12-fe689c87a8bf admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:45.513651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb7922-48f4-4340-8bfd-7f1d571153b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:45.515105 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb7922-48f4-4340-8bfd-7f1d571153b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.515488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb7922-48f4-4340-8bfd-7f1d571153b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.515855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33eec4-aabc-4759-a991-34991b1a851b service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:45.516157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb7922-48f4-4340-8bfd-7f1d571153b9 admin admin] PATH_INFO: `/v3/projects/5999c0589e3747c898aa4b030ee7f5b4/users/bbe992d56ebc4d38a212d8b599a1e1e8/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.523266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e54ca2-ef23-4a00-b182-91a0f0177bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:45.524023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e54ca2-ef23-4a00-b182-91a0f0177bc4 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.524310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e54ca2-ef23-4a00-b182-91a0f0177bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.524586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e54ca2-ef23-4a00-b182-91a0f0177bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.537642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb7922-48f4-4340-8bfd-7f1d571153b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5999c0589e3747c898aa4b030ee7f5b4, user_id=bbe992d56ebc4d38a212d8b599a1e1e8, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:45.540628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb7922-48f4-4340-8bfd-7f1d571153b9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:45.598269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e54ca2-ef23-4a00-b182-91a0f0177bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:45.600183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e54ca2-ef23-4a00-b182-91a0f0177bc4 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:45.611384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 806/1623] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:45.638171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 818/1624] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:45 2022] PUT /identity/v3/projects/5999c0589e3747c898aa4b030ee7f5b4/users/bbe992d56ebc4d38a212d8b599a1e1e8/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:45.645286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e54ca2-ef23-4a00-b182-91a0f0177bc4 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:45.668410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e54ca2-ef23-4a00-b182-91a0f0177bc4 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:45.704917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:45.759062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bee23-4a5b-44c9-82da-6530227269af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:45.761486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bee23-4a5b-44c9-82da-6530227269af admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.762901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bee23-4a5b-44c9-82da-6530227269af admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.763267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bee23-4a5b-44c9-82da-6530227269af admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.765296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bee23-4a5b-44c9-82da-6530227269af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:45.772703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bee23-4a5b-44c9-82da-6530227269af admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:45.796426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 807/1625] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:45.797953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb7922-48f4-4340-8bfd-7f1d571153b9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:45.810537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb7922-48f4-4340-8bfd-7f1d571153b9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:45.817978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e80be1a-bc4b-4695-8a8f-7f1fa29613f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:45.818942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80be1a-bc4b-4695-8a8f-7f1fa29613f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.819410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80be1a-bc4b-4695-8a8f-7f1fa29613f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.819763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80be1a-bc4b-4695-8a8f-7f1fa29613f6 admin admin] PATH_INFO: `/v3/projects/5999c0589e3747c898aa4b030ee7f5b4/users/bbe992d56ebc4d38a212d8b599a1e1e8/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.822063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80be1a-bc4b-4695-8a8f-7f1fa29613f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5999c0589e3747c898aa4b030ee7f5b4, user_id=bbe992d56ebc4d38a212d8b599a1e1e8, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:45.824998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e80be1a-bc4b-4695-8a8f-7f1fa29613f6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:45.838800 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 819/1626] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:45 2022] PUT /identity/v3/projects/5999c0589e3747c898aa4b030ee7f5b4/users/bbe992d56ebc4d38a212d8b599a1e1e8/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:45.850915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec902b41-1df7-49db-bbf9-ba77cfa9f489 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.850915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec902b41-1df7-49db-bbf9-ba77cfa9f489 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.850915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec902b41-1df7-49db-bbf9-ba77cfa9f489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.895437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ec902b41-1df7-49db-bbf9-ba77cfa9f489 None None] MFA Rules not processed for user `bbe992d56ebc4d38a212d8b599a1e1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:45.897715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ec902b41-1df7-49db-bbf9-ba77cfa9f489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:45.965654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950754a-7ff3-45cc-b39c-3c499d7b4e5e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.971435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950754a-7ff3-45cc-b39c-3c499d7b4e5e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.971435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950754a-7ff3-45cc-b39c-3c499d7b4e5e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:45.971435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 820/1627] 10.222.0.41 () {54 vars in 912 bytes} [Sat May 28 03:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:35:45.980287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574d0b-d525-4060-909e-2863fd07595a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:45.980871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574d0b-d525-4060-909e-2863fd07595a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:45.981910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6574d0b-d525-4060-909e-2863fd07595a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:46.006245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e6574d0b-d525-4060-909e-2863fd07595a None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:46.007481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e6574d0b-d525-4060-909e-2863fd07595a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:46.018718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 808/1628] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:46.027562 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec902b41-1df7-49db-bbf9-ba77cfa9f489 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:46.088628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:46.122901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 821/1629] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:35:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:46.130750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec902b41-1df7-49db-bbf9-ba77cfa9f489 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:46.136653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30096b-1853-4dec-a57f-53969651d3a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:46.137137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30096b-1853-4dec-a57f-53969651d3a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:46.137569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30096b-1853-4dec-a57f-53969651d3a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:46.138081 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30096b-1853-4dec-a57f-53969651d3a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:46.153197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30096b-1853-4dec-a57f-53969651d3a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:46.155823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30096b-1853-4dec-a57f-53969651d3a5 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:46.170193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 809/1630] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:35:46.298115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6574d0b-d525-4060-909e-2863fd07595a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:46.318014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6574d0b-d525-4060-909e-2863fd07595a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:46.380850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac475a11-532a-4369-964d-52c6123cc821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:46.380850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac475a11-532a-4369-964d-52c6123cc821 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:46.380850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac475a11-532a-4369-964d-52c6123cc821 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:46.380850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac475a11-532a-4369-964d-52c6123cc821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:46.422679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac475a11-532a-4369-964d-52c6123cc821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:46.424145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac475a11-532a-4369-964d-52c6123cc821 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:46.470336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 822/1631] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:46.816729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30096b-1853-4dec-a57f-53969651d3a5 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:46.826837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30096b-1853-4dec-a57f-53969651d3a5 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:46.840649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ce258-0582-4734-97b5-0642964091c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:46.842347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ce258-0582-4734-97b5-0642964091c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:46.842638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ce258-0582-4734-97b5-0642964091c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:46.844957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ce258-0582-4734-97b5-0642964091c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:46.983762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:46.992772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ce258-0582-4734-97b5-0642964091c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:46.995105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ce258-0582-4734-97b5-0642964091c0 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:47.021158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 810/1632] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:48.187779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bcc5-8b57-4d6f-95f8-4a6e7db9fec7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:48.189322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bcc5-8b57-4d6f-95f8-4a6e7db9fec7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:48.190156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1bcc5-8b57-4d6f-95f8-4a6e7db9fec7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:48.192520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 823/1633] 10.222.0.41 () {54 vars in 961 bytes} [Sat May 28 03:35:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:35:48.202264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3828b6-5e5d-433d-86c2-e55e5be40f52 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:48.202787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3828b6-5e5d-433d-86c2-e55e5be40f52 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:48.202787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3828b6-5e5d-433d-86c2-e55e5be40f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:48.244912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9e3828b6-5e5d-433d-86c2-e55e5be40f52 None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:48.244912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9e3828b6-5e5d-433d-86c2-e55e5be40f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:48.266117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 811/1634] 10.222.0.41 () {58 vars in 1058 bytes} [Sat May 28 03:35:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:48.267325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef92c91-92a1-406f-a6f8-19090aa049c0 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:48.267629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef92c91-92a1-406f-a6f8-19090aa049c0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:48.268001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef92c91-92a1-406f-a6f8-19090aa049c0 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:48.270093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 824/1635] 10.222.0.41 () {54 vars in 912 bytes} [Sat May 28 03:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:35:48.277517 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188adc3-042f-4d73-84bb-5b7e732845a0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:48.278359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188adc3-042f-4d73-84bb-5b7e732845a0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:48.278359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188adc3-042f-4d73-84bb-5b7e732845a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:48.307624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8188adc3-042f-4d73-84bb-5b7e732845a0 None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:48.308848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8188adc3-042f-4d73-84bb-5b7e732845a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:48.356230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 812/1636] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:35:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:48.356633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef92c91-92a1-406f-a6f8-19090aa049c0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:48.377495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef92c91-92a1-406f-a6f8-19090aa049c0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:48.386306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a8c8-f3a9-47e0-ad52-3183448050a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:48.387039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a8c8-f3a9-47e0-ad52-3183448050a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:48.387398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a8c8-f3a9-47e0-ad52-3183448050a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:48.388015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a8c8-f3a9-47e0-ad52-3183448050a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:48.407273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84a8c8-f3a9-47e0-ad52-3183448050a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:48.408717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84a8c8-f3a9-47e0-ad52-3183448050a9 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:48.429148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 825/1637] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:49.046607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22f9d3-a561-40cf-8e10-84edc2167ba7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:49.046900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22f9d3-a561-40cf-8e10-84edc2167ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:49.047188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22f9d3-a561-40cf-8e10-84edc2167ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:49.097992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3f22f9d3-a561-40cf-8e10-84edc2167ba7 None None] MFA Rules not processed for user `46e48b861bc2498c9670de030c783f54`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:49.106371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3f22f9d3-a561-40cf-8e10-84edc2167ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:49.260865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 813/1638] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:49.267036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c05c3-e583-4019-bdb7-6feba61b3f70 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:49.267547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c05c3-e583-4019-bdb7-6feba61b3f70 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:49.268052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c05c3-e583-4019-bdb7-6feba61b3f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:49.300860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8a1c05c3-e583-4019-bdb7-6feba61b3f70 None None] MFA Rules not processed for user `46e48b861bc2498c9670de030c783f54`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:49.303228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8a1c05c3-e583-4019-bdb7-6feba61b3f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:49.330784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 826/1639] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:49.347262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22f9d3-a561-40cf-8e10-84edc2167ba7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:49.368025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22f9d3-a561-40cf-8e10-84edc2167ba7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:49.385249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c05f55-c2b2-481b-abcb-24279dae5d09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:49.385986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c05f55-c2b2-481b-abcb-24279dae5d09 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:49.386276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c05f55-c2b2-481b-abcb-24279dae5d09 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:49.386618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c05f55-c2b2-481b-abcb-24279dae5d09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:49.405935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c05f55-c2b2-481b-abcb-24279dae5d09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:49.407578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c05f55-c2b2-481b-abcb-24279dae5d09 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:49.446188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 814/1640] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:50.081310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d8a65-6452-47eb-b69d-6a2ddcd3c1c4 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:50.081670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d8a65-6452-47eb-b69d-6a2ddcd3c1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:50.081960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d8a65-6452-47eb-b69d-6a2ddcd3c1c4 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:50.084389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 827/1641] 10.222.0.41 () {54 vars in 912 bytes} [Sat May 28 03:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:35:50.090634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62835438-e7a0-4f2d-9fdb-2912ea128174 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:50.090934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62835438-e7a0-4f2d-9fdb-2912ea128174 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:50.096521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62835438-e7a0-4f2d-9fdb-2912ea128174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:50.170654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-62835438-e7a0-4f2d-9fdb-2912ea128174 None None] MFA Rules not processed for user `1dd9a0e75308460080902d1da74074c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:50.171898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-62835438-e7a0-4f2d-9fdb-2912ea128174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:50.327327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 815/1642] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:35:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:50.434334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d8a65-6452-47eb-b69d-6a2ddcd3c1c4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:50.448311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d8a65-6452-47eb-b69d-6a2ddcd3c1c4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:50.452746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1a52-f50c-40fc-8e5d-1f427cee566f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:50.454195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1a52-f50c-40fc-8e5d-1f427cee566f service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:50.454487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1a52-f50c-40fc-8e5d-1f427cee566f service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:50.454763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1a52-f50c-40fc-8e5d-1f427cee566f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:50.466064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1a52-f50c-40fc-8e5d-1f427cee566f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:50.467882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1a52-f50c-40fc-8e5d-1f427cee566f service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:50.483570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 828/1643] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:51.662526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10c8df-a670-4eed-8b89-59a542f5e803 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:51.663502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10c8df-a670-4eed-8b89-59a542f5e803 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:51.664136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10c8df-a670-4eed-8b89-59a542f5e803 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:51.666043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 816/1644] 10.222.0.41 () {54 vars in 960 bytes} [Sat May 28 03:35:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:35:51.675023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1c4f2-b40c-447f-a9b9-1502adf681d7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:51.676542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1c4f2-b40c-447f-a9b9-1502adf681d7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:51.676965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1c4f2-b40c-447f-a9b9-1502adf681d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:51.714130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-29b1c4f2-b40c-447f-a9b9-1502adf681d7 None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:51.715574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-29b1c4f2-b40c-447f-a9b9-1502adf681d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:51.749246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e064c-2695-4c70-9df5-53508e0094dc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:51.752520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e064c-2695-4c70-9df5-53508e0094dc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:51.753175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e064c-2695-4c70-9df5-53508e0094dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:51.818664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-177e064c-2695-4c70-9df5-53508e0094dc None None] MFA Rules not processed for user `e0eb0ab5585747d098bbb0b18770f4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:51.821706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-177e064c-2695-4c70-9df5-53508e0094dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:51.899475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 829/1645] 10.222.0.41 () {58 vars in 1057 bytes} [Sat May 28 03:35:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:51.901885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1c4f2-b40c-447f-a9b9-1502adf681d7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:51.917804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1c4f2-b40c-447f-a9b9-1502adf681d7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:51.927130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:51.928857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:51.930797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:51.931573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:51.936051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:51.951738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:51.975768 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:51.975768 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:51.980944 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 830/1646] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:52.012757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 817/1647] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:52.028723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:52.050850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769406be-adca-47ca-9c6b-a28c9686390d service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:52.067600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081c1e5-767e-4151-86a3-8cebc1e81db0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:52.083656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c1e5-767e-4151-86a3-8cebc1e81db0 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:52.084457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c1e5-767e-4151-86a3-8cebc1e81db0 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:52.084950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081c1e5-767e-4151-86a3-8cebc1e81db0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:52.123633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081c1e5-767e-4151-86a3-8cebc1e81db0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:52.125303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081c1e5-767e-4151-86a3-8cebc1e81db0 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:52.183132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e064c-2695-4c70-9df5-53508e0094dc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:52.195193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 831/1648] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:52.376053 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e064c-2695-4c70-9df5-53508e0094dc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:52.389743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:52.391201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:52.392706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:52.394675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] PATH_INFO: `/v3/users/4c9b89c578a34597b143cb9294946300` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:52.399261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9b89c578a34597b143cb9294946300)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:52.402167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: WARNING py.warnings [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:35:52.402167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: warnings.warn(msg) May 28 03:35:52.402167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: May 28 03:35:52.406385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:52.790219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9b89c578a34597b143cb9294946300'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:52.852918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9b89c578a34597b143cb9294946300'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:52.854426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9b89c578a34597b143cb9294946300'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:52.857626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9b89c578a34597b143cb9294946300'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:52.977597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9b89c578a34597b143cb9294946300'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:52.998075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 818/1649] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:35:52 2022] DELETE /identity/v3/users/4c9b89c578a34597b143cb9294946300 => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:53.003218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda69a-dc3a-48b1-9ed0-fe1b50f25749 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:53.003218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda69a-dc3a-48b1-9ed0-fe1b50f25749 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:53.004355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda69a-dc3a-48b1-9ed0-fe1b50f25749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:53.059538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-abbda69a-dc3a-48b1-9ed0-fe1b50f25749 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:53.066696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-abbda69a-dc3a-48b1-9ed0-fe1b50f25749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:53.214243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 832/1650] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:53.228905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:53.231249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:53.392004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca71279-94f4-438d-93dd-63d01c00d848 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:53.398160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1ee7-0cd7-43a1-adc7-778121a9f31c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:53.398828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1ee7-0cd7-43a1-adc7-778121a9f31c service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:53.399211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1ee7-0cd7-43a1-adc7-778121a9f31c service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:53.399495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1ee7-0cd7-43a1-adc7-778121a9f31c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:53.415911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1ee7-0cd7-43a1-adc7-778121a9f31c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:53.415911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1ee7-0cd7-43a1-adc7-778121a9f31c service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:53.494373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 819/1651] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:54.613796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbda69a-dc3a-48b1-9ed0-fe1b50f25749 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:54.616236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-abbda69a-dc3a-48b1-9ed0-fe1b50f25749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:54.637352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbda69a-dc3a-48b1-9ed0-fe1b50f25749 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:54.649426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:54.667260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:54.668093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:54.669183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] PATH_INFO: `/v3/projects/c547fb861a0e4888b6bef6c30c5f0c73` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:54.671332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=c547fb861a0e4888b6bef6c30c5f0c73)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:54.672661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 03:35:54.672661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: warnings.warn(msg) May 28 03:35:54.672661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: May 28 03:35:54.675587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:54.687673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:54.854840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c547fb861a0e4888b6bef6c30c5f0c73', 'request_id': 'req-b467d24d-51fa-4b5e-bc77-6f3265df90ab'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:54.875002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c547fb861a0e4888b6bef6c30c5f0c73', 'request_id': 'req-b467d24d-51fa-4b5e-bc77-6f3265df90ab'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:54.890975 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c547fb861a0e4888b6bef6c30c5f0c73', 'request_id': 'req-b467d24d-51fa-4b5e-bc77-6f3265df90ab'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:55.000914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] The token cache is being invalidate because project c547fb861a0e4888b6bef6c30c5f0c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:35:55.001563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:55.002038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b467d24d-51fa-4b5e-bc77-6f3265df90ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:35:55.006360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 833/1652] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:35:54 2022] DELETE /identity/v3/projects/c547fb861a0e4888b6bef6c30c5f0c73 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:55.012978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd5af-bf20-4ec6-97dc-9f9a105df09f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:55.014374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd5af-bf20-4ec6-97dc-9f9a105df09f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:55.015040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acd5af-bf20-4ec6-97dc-9f9a105df09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:55.030708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:55.057875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-92acd5af-bf20-4ec6-97dc-9f9a105df09f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:55.059359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-92acd5af-bf20-4ec6-97dc-9f9a105df09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:55.215921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 820/1653] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:55.218574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871c7d-1492-4a4d-8c31-84e87763a0cc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:55.220576 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871c7d-1492-4a4d-8c31-84e87763a0cc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:55.221573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63871c7d-1492-4a4d-8c31-84e87763a0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:55.280715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-63871c7d-1492-4a4d-8c31-84e87763a0cc None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:55.287602 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-63871c7d-1492-4a4d-8c31-84e87763a0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:55.321639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 834/1654] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:55.326032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd5af-bf20-4ec6-97dc-9f9a105df09f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:55.347375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acd5af-bf20-4ec6-97dc-9f9a105df09f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:55.369905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:55.370841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:55.371324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:55.371742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:55.373356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:35:55.374853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:55.380217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:55.415896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 821/1655] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:35:55.417046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63871c7d-1492-4a4d-8c31-84e87763a0cc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:55.430086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63871c7d-1492-4a4d-8c31-84e87763a0cc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:55.450576 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583bfc-7596-47df-9cb8-e6e8ebb339d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:55.451257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583bfc-7596-47df-9cb8-e6e8ebb339d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:55.451547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583bfc-7596-47df-9cb8-e6e8ebb339d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:55.451831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58583bfc-7596-47df-9cb8-e6e8ebb339d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:55.452533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583bfc-7596-47df-9cb8-e6e8ebb339d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:55.454502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58583bfc-7596-47df-9cb8-e6e8ebb339d4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:55.591980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 835/1656] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:55 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:55.593238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:55.613828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2270f-eda9-48eb-b9c9-52c98e4a7bc2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:55.786828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3faa31-1462-49aa-a584-a325cee6ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:55.787538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3faa31-1462-49aa-a584-a325cee6ea97 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:55.787823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3faa31-1462-49aa-a584-a325cee6ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:55.788102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3faa31-1462-49aa-a584-a325cee6ea97 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:55.790789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3faa31-1462-49aa-a584-a325cee6ea97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:55.794692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3faa31-1462-49aa-a584-a325cee6ea97 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:55.897449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 822/1657] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:55 2022] POST /identity/v3/users => generated 288 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:55.901896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583bfc-7596-47df-9cb8-e6e8ebb339d4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:55.921670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58583bfc-7596-47df-9cb8-e6e8ebb339d4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:55.959266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f83d7-0f62-4508-9ed6-1852273e65dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:55.960228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f83d7-0f62-4508-9ed6-1852273e65dd admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:55.960521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f83d7-0f62-4508-9ed6-1852273e65dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:55.960805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f83d7-0f62-4508-9ed6-1852273e65dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:55.961604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f83d7-0f62-4508-9ed6-1852273e65dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:55.963260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f83d7-0f62-4508-9ed6-1852273e65dd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:55.989487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 836/1658] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:55.993689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3faa31-1462-49aa-a584-a325cee6ea97 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:56.013626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3faa31-1462-49aa-a584-a325cee6ea97 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:56.021657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d468fc-db4b-478b-93ca-62b7f804cf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:56.022414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d468fc-db4b-478b-93ca-62b7f804cf64 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:56.022705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d468fc-db4b-478b-93ca-62b7f804cf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:56.022984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d468fc-db4b-478b-93ca-62b7f804cf64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:56.023713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d468fc-db4b-478b-93ca-62b7f804cf64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:56.025564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d468fc-db4b-478b-93ca-62b7f804cf64 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:56.054603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 823/1659] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:56.059079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f83d7-0f62-4508-9ed6-1852273e65dd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:56.074590 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f83d7-0f62-4508-9ed6-1852273e65dd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:56.089736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a21a-fabc-4fd6-b487-24f1dfb77307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:56.090650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a21a-fabc-4fd6-b487-24f1dfb77307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:56.091092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a21a-fabc-4fd6-b487-24f1dfb77307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:56.091512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1a21a-fabc-4fd6-b487-24f1dfb77307 admin admin] PATH_INFO: `/v3/projects/9e879c61f5ef4544b2f2774817e19694/users/c5f14a3e9af24525a940cd2b0a047830/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:56.123194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1a21a-fabc-4fd6-b487-24f1dfb77307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e879c61f5ef4544b2f2774817e19694, user_id=c5f14a3e9af24525a940cd2b0a047830, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:56.129093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1a21a-fabc-4fd6-b487-24f1dfb77307 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:56.269935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 837/1660] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:56 2022] PUT /identity/v3/projects/9e879c61f5ef4544b2f2774817e19694/users/c5f14a3e9af24525a940cd2b0a047830/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:56.277495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ec7f-d278-44e1-8e20-5c2872fbe023 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:56.277978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ec7f-d278-44e1-8e20-5c2872fbe023 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:56.278556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ec7f-d278-44e1-8e20-5c2872fbe023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:56.324250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2df9ec7f-d278-44e1-8e20-5c2872fbe023 None None] MFA Rules not processed for user `c5f14a3e9af24525a940cd2b0a047830`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:56.325828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2df9ec7f-d278-44e1-8e20-5c2872fbe023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:56.446990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 824/1661] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:56.449847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a21a-fabc-4fd6-b487-24f1dfb77307 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:56.620852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1a21a-fabc-4fd6-b487-24f1dfb77307 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:56.627735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d7c69-07ef-4165-9c5a-e0a002aa0d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:56.629281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d7c69-07ef-4165-9c5a-e0a002aa0d3f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:56.629710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d7c69-07ef-4165-9c5a-e0a002aa0d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:56.630259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d7c69-07ef-4165-9c5a-e0a002aa0d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:56.631107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d7c69-07ef-4165-9c5a-e0a002aa0d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:56.633727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d7c69-07ef-4165-9c5a-e0a002aa0d3f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:56.663424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:56.760513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 838/1662] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:35:56 2022] POST /identity/v3/projects => generated 375 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:35:56.764717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ec7f-d278-44e1-8e20-5c2872fbe023 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:56.785601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ec7f-d278-44e1-8e20-5c2872fbe023 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:56.919821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e72d78-d380-43a7-a1fb-f64a49af009f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:56.921957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72d78-d380-43a7-a1fb-f64a49af009f admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:56.922433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72d78-d380-43a7-a1fb-f64a49af009f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:56.922850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72d78-d380-43a7-a1fb-f64a49af009f admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:56.923764 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e72d78-d380-43a7-a1fb-f64a49af009f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:56.931567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e72d78-d380-43a7-a1fb-f64a49af009f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:57.062139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 825/1663] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:35:56 2022] POST /identity/v3/users => generated 294 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:35:57.068372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d7c69-07ef-4165-9c5a-e0a002aa0d3f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:57.087618 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d7c69-07ef-4165-9c5a-e0a002aa0d3f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:57.104942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d6517-feb9-4360-b165-090ade31fffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:57.106761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6517-feb9-4360-b165-090ade31fffa admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:57.108368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6517-feb9-4360-b165-090ade31fffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:57.109154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d6517-feb9-4360-b165-090ade31fffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:57.124282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d6517-feb9-4360-b165-090ade31fffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:57.127518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d6517-feb9-4360-b165-090ade31fffa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:57.146553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 839/1664] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:57.150559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e72d78-d380-43a7-a1fb-f64a49af009f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:57.168129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e72d78-d380-43a7-a1fb-f64a49af009f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:57.184171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abd977-a6d0-4a09-bf81-f0f050045965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:57.185451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd977-a6d0-4a09-bf81-f0f050045965 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:57.185880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd977-a6d0-4a09-bf81-f0f050045965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:57.186327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abd977-a6d0-4a09-bf81-f0f050045965 admin admin] PATH_INFO: `/v3/projects/223b1d493a044cdc9d1718eb9f7e2f20/users/47bb4ac989ce4d0f8a75fd382d1addf2/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:57.224786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abd977-a6d0-4a09-bf81-f0f050045965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223b1d493a044cdc9d1718eb9f7e2f20, user_id=47bb4ac989ce4d0f8a75fd382d1addf2, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:57.228003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abd977-a6d0-4a09-bf81-f0f050045965 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:57.348646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 826/1665] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:57 2022] PUT /identity/v3/projects/223b1d493a044cdc9d1718eb9f7e2f20/users/47bb4ac989ce4d0f8a75fd382d1addf2/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:35:57.352609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d6517-feb9-4360-b165-090ade31fffa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:57.368529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d6517-feb9-4360-b165-090ade31fffa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:57.377051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:57.502194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb5707-9f61-4fa8-abfe-9981adcbcfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:57.503121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5707-9f61-4fa8-abfe-9981adcbcfe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:57.504109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5707-9f61-4fa8-abfe-9981adcbcfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:57.514121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5707-9f61-4fa8-abfe-9981adcbcfe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:57.515282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb5707-9f61-4fa8-abfe-9981adcbcfe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:57.522585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb5707-9f61-4fa8-abfe-9981adcbcfe7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:57.543892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abd977-a6d0-4a09-bf81-f0f050045965 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:57.544014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 840/1666] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:35:57.560891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abd977-a6d0-4a09-bf81-f0f050045965 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:57.567859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:57.569224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:57.569880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:57.570362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] PATH_INFO: `/v3/projects/223b1d493a044cdc9d1718eb9f7e2f20/users/47bb4ac989ce4d0f8a75fd382d1addf2/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:57.574001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223b1d493a044cdc9d1718eb9f7e2f20, user_id=47bb4ac989ce4d0f8a75fd382d1addf2, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:57.577559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:57.611559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d2ca-c990-497e-9271-2af1a7266b02 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:57.612127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d2ca-c990-497e-9271-2af1a7266b02 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:57.612592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6d2ca-c990-497e-9271-2af1a7266b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:57.615220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 827/1667] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:35:57 2022] PUT /identity/v3/projects/223b1d493a044cdc9d1718eb9f7e2f20/users/47bb4ac989ce4d0f8a75fd382d1addf2/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:35:57.652107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f7c6d2ca-c990-497e-9271-2af1a7266b02 None None] MFA Rules not processed for user `47bb4ac989ce4d0f8a75fd382d1addf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:57.653585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f7c6d2ca-c990-497e-9271-2af1a7266b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:57.886487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 841/1668] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:57.909075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:57.911826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:58.413372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15a2b7-ff38-4ac1-af5c-732b322e37b0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:58.434623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58dc23-17be-43a7-879a-3716d72c408c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:58.435311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58dc23-17be-43a7-879a-3716d72c408c service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:58.435653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58dc23-17be-43a7-879a-3716d72c408c service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:58.437490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58dc23-17be-43a7-879a-3716d72c408c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:58.496056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043c3b-eba4-4dce-a573-6a8eb97c4fa4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:58.497192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043c3b-eba4-4dce-a573-6a8eb97c4fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:58.497646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043c3b-eba4-4dce-a573-6a8eb97c4fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:58.550578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d4043c3b-eba4-4dce-a573-6a8eb97c4fa4 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:58.551830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4043c3b-eba4-4dce-a573-6a8eb97c4fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:58.630695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58dc23-17be-43a7-879a-3716d72c408c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:58.637440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58dc23-17be-43a7-879a-3716d72c408c service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:58.675881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 828/1669] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:35:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:35:58.725811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 842/1670] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:35:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:35:58.785967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77106-7c1d-428a-9805-4b96966d71d8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:58.786070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77106-7c1d-428a-9805-4b96966d71d8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:58.786290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf77106-7c1d-428a-9805-4b96966d71d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:58.820370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:58.847278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5cf77106-7c1d-428a-9805-4b96966d71d8 None None] MFA Rules not processed for user `e0eb0ab5585747d098bbb0b18770f4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:35:58.850292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5cf77106-7c1d-428a-9805-4b96966d71d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:59.104840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 829/1671] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:35:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:35:59.359565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4043c3b-eba4-4dce-a573-6a8eb97c4fa4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:59.364718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4043c3b-eba4-4dce-a573-6a8eb97c4fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:59.374656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77106-7c1d-428a-9805-4b96966d71d8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:35:59.377558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5cf77106-7c1d-428a-9805-4b96966d71d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:35:59.563385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4043c3b-eba4-4dce-a573-6a8eb97c4fa4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:59.574473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf77106-7c1d-428a-9805-4b96966d71d8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:35:59.575697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccbdc7-7559-4bf5-b771-d377534971e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:59.576483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccbdc7-7559-4bf5-b771-d377534971e0 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:59.578945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccbdc7-7559-4bf5-b771-d377534971e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:59.579804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccbdc7-7559-4bf5-b771-d377534971e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:59.590382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:35:59.596983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:35:59.596983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:35:59.596983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] PATH_INFO: `/v3/users/cbea6fa382544157b25628d3446fa3f8` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:35:59.597515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccbdc7-7559-4bf5-b771-d377534971e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:59.597515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccbdc7-7559-4bf5-b771-d377534971e0 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:59.598446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbea6fa382544157b25628d3446fa3f8)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:35:59.604241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:35:59.613084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 843/1672] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:35:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:35:59.792651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:35:59.961933 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbea6fa382544157b25628d3446fa3f8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:00.088185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbea6fa382544157b25628d3446fa3f8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:00.094961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbea6fa382544157b25628d3446fa3f8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:00.112176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbea6fa382544157b25628d3446fa3f8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:00.262006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbea6fa382544157b25628d3446fa3f8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:00.315025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 830/1673] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:35:59 2022] DELETE /identity/v3/users/cbea6fa382544157b25628d3446fa3f8 => generated 0 bytes in 941 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:00.326041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa63c53-147c-45ff-8f5d-fbc6c47780f9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:00.326854 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa63c53-147c-45ff-8f5d-fbc6c47780f9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:00.327746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa63c53-147c-45ff-8f5d-fbc6c47780f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:00.403787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-afa63c53-147c-45ff-8f5d-fbc6c47780f9 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:00.421161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-afa63c53-147c-45ff-8f5d-fbc6c47780f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:00.702113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 844/1674] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:01.012278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:01.013985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:01.163460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:01.400707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844626c4-e468-41cf-a136-ab6fc1bf54d7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:01.400707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:01.400707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:01.400707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:01.402130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:01.402130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:01.402130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:01.412414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 831/1675] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:01.785642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741bfaa-78eb-4e74-b04a-310278ef8abf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:01.786905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741bfaa-78eb-4e74-b04a-310278ef8abf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:01.787764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741bfaa-78eb-4e74-b04a-310278ef8abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:01.933030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b741bfaa-78eb-4e74-b04a-310278ef8abf None None] MFA Rules not processed for user `9d420e4b554e4deab21b89094e0547ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:01.945492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b741bfaa-78eb-4e74-b04a-310278ef8abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:02.158605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 845/1676] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:02.177197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:02.185453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:02.435966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf915b-8811-4522-b160-0026b87a1470 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:02.471579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2823-53dd-47e5-bc70-87531af2e0dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:02.472882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2823-53dd-47e5-bc70-87531af2e0dc service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:02.473133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2823-53dd-47e5-bc70-87531af2e0dc service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:02.473430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2823-53dd-47e5-bc70-87531af2e0dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:02.518616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2823-53dd-47e5-bc70-87531af2e0dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:02.520777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2823-53dd-47e5-bc70-87531af2e0dc service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:02.545382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741bfaa-78eb-4e74-b04a-310278ef8abf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:02.545382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b741bfaa-78eb-4e74-b04a-310278ef8abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:02.574290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 832/1677] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:02.600782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741bfaa-78eb-4e74-b04a-310278ef8abf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:02.623840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:02.645200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:02.648545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:02.658715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] PATH_INFO: `/v3/projects/d9bf41aa6e93483c9c73351cefaf7f38` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:02.660682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9bf41aa6e93483c9c73351cefaf7f38)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:02.685189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:02.814298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea231aad-dcb5-44da-ab0a-567e896c9d95 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:02.814842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea231aad-dcb5-44da-ab0a-567e896c9d95 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:02.815289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea231aad-dcb5-44da-ab0a-567e896c9d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:02.853598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ea231aad-dcb5-44da-ab0a-567e896c9d95 None None] MFA Rules not processed for user `dc6a3335562448d1934fa18e8f51b119`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:02.858891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ea231aad-dcb5-44da-ab0a-567e896c9d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:02.966313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:02.968845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9bf41aa6e93483c9c73351cefaf7f38', 'request_id': 'req-702b857c-5914-4748-a31d-10af585e85b3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:02.981462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9bf41aa6e93483c9c73351cefaf7f38', 'request_id': 'req-702b857c-5914-4748-a31d-10af585e85b3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:02.990531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9bf41aa6e93483c9c73351cefaf7f38', 'request_id': 'req-702b857c-5914-4748-a31d-10af585e85b3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:03.139365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] The token cache is being invalidate because project d9bf41aa6e93483c9c73351cefaf7f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:03.139904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:03.142314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702b857c-5914-4748-a31d-10af585e85b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:03.153924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 846/1678] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:02 2022] DELETE /identity/v3/projects/d9bf41aa6e93483c9c73351cefaf7f38 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:03.161798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a455e-e650-4d80-8587-619503ff923a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:03.163496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a455e-e650-4d80-8587-619503ff923a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:03.165082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a455e-e650-4d80-8587-619503ff923a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:03.189202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 833/1679] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:03.214643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea231aad-dcb5-44da-ab0a-567e896c9d95 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:03.239162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e22a455e-e650-4d80-8587-619503ff923a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:03.247099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e22a455e-e650-4d80-8587-619503ff923a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:03.258838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea231aad-dcb5-44da-ab0a-567e896c9d95 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:03.397515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:03.398420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:03.398906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:03.399339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:03.401787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:03.499597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 847/1680] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:03.515585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41871fe3-ce3e-4f1a-8076-ad28ad014b72 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:03.515585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41871fe3-ce3e-4f1a-8076-ad28ad014b72 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:03.515585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41871fe3-ce3e-4f1a-8076-ad28ad014b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:03.568041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-41871fe3-ce3e-4f1a-8076-ad28ad014b72 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:03.574287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-41871fe3-ce3e-4f1a-8076-ad28ad014b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:03.604579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:03.610554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 848/1681] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:03.612655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41871fe3-ce3e-4f1a-8076-ad28ad014b72 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:03.614917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:03.654218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41871fe3-ce3e-4f1a-8076-ad28ad014b72 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:03.660434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 834/1682] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:03.674177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:03.674930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:03.675838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:03.676059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:03.679722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:03.680070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:03.687270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:03.724109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 849/1683] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:36:03.728574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:03.749373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c5136-6ad2-44cf-82c1-a5fb0716bad6 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:03.765951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:03.775043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:03.775863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:03.776248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:03.776628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:03.778126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:03.780520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:03.997002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 835/1684] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:03 2022] POST /identity/v3/projects => generated 369 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:03.997973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:04.018571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c27b42-3886-4ce7-bac9-9f3dc292e26e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:04.264302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966c79a-f3d0-4a70-bca6-83fb65e31102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:04.267053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c79a-f3d0-4a70-bca6-83fb65e31102 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:04.267364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c79a-f3d0-4a70-bca6-83fb65e31102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:04.267628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966c79a-f3d0-4a70-bca6-83fb65e31102 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:04.268345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966c79a-f3d0-4a70-bca6-83fb65e31102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:04.274453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966c79a-f3d0-4a70-bca6-83fb65e31102 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:04.397151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:04.401016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 850/1685] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:03 2022] POST /identity/v3/users => generated 285 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:04.401261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:04.404560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966c79a-f3d0-4a70-bca6-83fb65e31102 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:04.432621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966c79a-f3d0-4a70-bca6-83fb65e31102 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:04.461610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ebffc-5577-4c59-ad5e-d1227459d178 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:04.475249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc11aa-efe3-4dbc-af5e-1579ce339d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:04.476363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc11aa-efe3-4dbc-af5e-1579ce339d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:04.476363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc11aa-efe3-4dbc-af5e-1579ce339d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:04.476558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc11aa-efe3-4dbc-af5e-1579ce339d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:04.477644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc11aa-efe3-4dbc-af5e-1579ce339d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:04.480125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc11aa-efe3-4dbc-af5e-1579ce339d9c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:04.482774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:04.485695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:04.485991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:04.488347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] PATH_INFO: `/v3/users/9b9ccd945d4b4e72bbd47c7026ef041a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:04.493479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9ccd945d4b4e72bbd47c7026ef041a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:04.506251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:04.530827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 851/1686] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:04.530827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc11aa-efe3-4dbc-af5e-1579ce339d9c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:04.556396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc11aa-efe3-4dbc-af5e-1579ce339d9c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:04.574160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296a3a3-727e-493a-8224-cf390285ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:04.576017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a3a3-727e-493a-8224-cf390285ff5e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:04.577707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a3a3-727e-493a-8224-cf390285ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:04.578126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296a3a3-727e-493a-8224-cf390285ff5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:04.578992 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296a3a3-727e-493a-8224-cf390285ff5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:04.590208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296a3a3-727e-493a-8224-cf390285ff5e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:04.660593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 852/1687] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:04.662901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296a3a3-727e-493a-8224-cf390285ff5e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:04.691066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296a3a3-727e-493a-8224-cf390285ff5e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:04.724979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14e237-4cf8-49b9-a32a-f988f4815426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:04.726980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14e237-4cf8-49b9-a32a-f988f4815426 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:04.742365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14e237-4cf8-49b9-a32a-f988f4815426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:04.744966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14e237-4cf8-49b9-a32a-f988f4815426 admin admin] PATH_INFO: `/v3/projects/47d1882059f848cbaf38514295d3fcdd/users/8c60d488b0da4fffb7aa52d4ce900f5f/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:04.800652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14e237-4cf8-49b9-a32a-f988f4815426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d1882059f848cbaf38514295d3fcdd, user_id=8c60d488b0da4fffb7aa52d4ce900f5f, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:04.803420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14e237-4cf8-49b9-a32a-f988f4815426 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:04.910510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9ccd945d4b4e72bbd47c7026ef041a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:04.913283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:04.919773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 853/1688] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:04 2022] PUT /identity/v3/projects/47d1882059f848cbaf38514295d3fcdd/users/8c60d488b0da4fffb7aa52d4ce900f5f/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:04.921435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a17ef-fd29-4062-b5a4-d8a40d6edf1d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:04.921723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a17ef-fd29-4062-b5a4-d8a40d6edf1d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:04.922475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a17ef-fd29-4062-b5a4-d8a40d6edf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:04.957398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9ccd945d4b4e72bbd47c7026ef041a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:04.959168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9ccd945d4b4e72bbd47c7026ef041a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:04.976498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9ccd945d4b4e72bbd47c7026ef041a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:04.990611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-947a17ef-fd29-4062-b5a4-d8a40d6edf1d None None] MFA Rules not processed for user `8c60d488b0da4fffb7aa52d4ce900f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:04.993134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-947a17ef-fd29-4062-b5a4-d8a40d6edf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:05.056336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30bf588a-79a6-4794-924c-4a8ba0c79b43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9ccd945d4b4e72bbd47c7026ef041a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:05.132063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 836/1689] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:04 2022] DELETE /identity/v3/users/9b9ccd945d4b4e72bbd47c7026ef041a => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:05.138676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3f08c-11d5-46d2-8695-d518ab8e663b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:05.139322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3f08c-11d5-46d2-8695-d518ab8e663b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:05.139849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3f08c-11d5-46d2-8695-d518ab8e663b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:05.170664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 854/1690] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:05.174424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9fe3f08c-11d5-46d2-8695-d518ab8e663b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:05.175683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a17ef-fd29-4062-b5a4-d8a40d6edf1d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:05.177070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9fe3f08c-11d5-46d2-8695-d518ab8e663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:05.179851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-947a17ef-fd29-4062-b5a4-d8a40d6edf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:05.318083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:05.431214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 837/1691] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:05.445590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3f08c-11d5-46d2-8695-d518ab8e663b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:05.447654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9fe3f08c-11d5-46d2-8695-d518ab8e663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:05.493026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a17ef-fd29-4062-b5a4-d8a40d6edf1d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:05.508833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:05.510006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:05.513036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:05.513036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:05.513036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:05.516553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:05.584508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3f08c-11d5-46d2-8695-d518ab8e663b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:05.590147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03651fa4-e770-4fe1-bb00-72333e722327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:05.591156 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03651fa4-e770-4fe1-bb00-72333e722327 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:05.591644 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03651fa4-e770-4fe1-bb00-72333e722327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:05.592135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03651fa4-e770-4fe1-bb00-72333e722327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:05.606135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03651fa4-e770-4fe1-bb00-72333e722327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:05.608378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03651fa4-e770-4fe1-bb00-72333e722327 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:05.634359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 838/1692] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:05.636001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b1086-da09-427f-9a86-14ff4f90fd34 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:05.636627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b1086-da09-427f-9a86-14ff4f90fd34 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:05.637265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b1086-da09-427f-9a86-14ff4f90fd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:05.661630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 855/1693] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:05 2022] POST /identity/v3/projects => generated 369 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:05.662865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:05.667948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:05.711155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e14b1086-da09-427f-9a86-14ff4f90fd34 None None] MFA Rules not processed for user `1dd9a0e75308460080902d1da74074c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:05.716225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e14b1086-da09-427f-9a86-14ff4f90fd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:05.818558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090a0bc-6518-4033-b905-2f1724c1ea00 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:05.822509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:05.823228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:05.823520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:05.823799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:05.834692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:05.850708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 839/1694] 10.222.0.27 () {58 vars in 1015 bytes} [Sat May 28 03:36:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:05.852323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b1086-da09-427f-9a86-14ff4f90fd34 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:06.033568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:06.036419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:06.058056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 856/1695] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:06.178458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b1086-da09-427f-9a86-14ff4f90fd34 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:06.222792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed4722-4429-4129-a220-f9f1186f9821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:06.223492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed4722-4429-4129-a220-f9f1186f9821 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:06.223775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed4722-4429-4129-a220-f9f1186f9821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:06.224050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed4722-4429-4129-a220-f9f1186f9821 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:06.267792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed4722-4429-4129-a220-f9f1186f9821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:06.297618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed4722-4429-4129-a220-f9f1186f9821 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:06.593524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:06.615880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b50ad-4ba4-4a13-a6e8-77c7663108fb service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:06.631504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9822f66-8707-45ed-8bca-a9397b6ea525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:06.631504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9822f66-8707-45ed-8bca-a9397b6ea525 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:06.638595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9822f66-8707-45ed-8bca-a9397b6ea525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:06.639245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9822f66-8707-45ed-8bca-a9397b6ea525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:06.655242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9822f66-8707-45ed-8bca-a9397b6ea525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:06.667371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9822f66-8707-45ed-8bca-a9397b6ea525 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:06.667566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 840/1696] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:05 2022] POST /identity/v3/users => generated 291 bytes in 816 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:06.672502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed4722-4429-4129-a220-f9f1186f9821 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:06.686550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 857/1697] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:06.692074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed4722-4429-4129-a220-f9f1186f9821 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:06.704354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f3a26-08d7-4564-a95c-4a9e082529a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:06.717735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f3a26-08d7-4564-a95c-4a9e082529a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:06.727000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f3a26-08d7-4564-a95c-4a9e082529a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:06.727572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f3a26-08d7-4564-a95c-4a9e082529a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:06.728945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f3a26-08d7-4564-a95c-4a9e082529a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:06.731571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f3a26-08d7-4564-a95c-4a9e082529a3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:06.808756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 841/1698] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:06.812870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9822f66-8707-45ed-8bca-a9397b6ea525 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:06.859844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9822f66-8707-45ed-8bca-a9397b6ea525 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:06.879679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80f9dc-8b6b-48ab-9489-b91bd53796d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:06.880745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80f9dc-8b6b-48ab-9489-b91bd53796d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:06.883964 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80f9dc-8b6b-48ab-9489-b91bd53796d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:06.884438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80f9dc-8b6b-48ab-9489-b91bd53796d9 admin admin] PATH_INFO: `/v3/projects/7ea817fa1c264c0a9921b57c4501266d/users/0d0936c8c2a944e09109c46b129a95c8/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:06.960477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80f9dc-8b6b-48ab-9489-b91bd53796d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea817fa1c264c0a9921b57c4501266d, user_id=0d0936c8c2a944e09109c46b129a95c8, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:06.973633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80f9dc-8b6b-48ab-9489-b91bd53796d9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:07.106125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 858/1699] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:06 2022] PUT /identity/v3/projects/7ea817fa1c264c0a9921b57c4501266d/users/0d0936c8c2a944e09109c46b129a95c8/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:07.110446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f3a26-08d7-4564-a95c-4a9e082529a3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:07.129301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f3a26-08d7-4564-a95c-4a9e082529a3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:07.318696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4559fba-50f3-4fe4-9e7f-797d8d3e2569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:07.319987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4559fba-50f3-4fe4-9e7f-797d8d3e2569 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:07.319987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4559fba-50f3-4fe4-9e7f-797d8d3e2569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:07.323367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4559fba-50f3-4fe4-9e7f-797d8d3e2569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:07.325439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4559fba-50f3-4fe4-9e7f-797d8d3e2569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:07.330866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4559fba-50f3-4fe4-9e7f-797d8d3e2569 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:07.356729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 842/1700] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:07.365632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80f9dc-8b6b-48ab-9489-b91bd53796d9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:07.405644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80f9dc-8b6b-48ab-9489-b91bd53796d9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:07.425817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff60859-59c4-4099-8d56-91c66f043f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:07.428782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff60859-59c4-4099-8d56-91c66f043f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:07.430395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff60859-59c4-4099-8d56-91c66f043f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:07.431565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff60859-59c4-4099-8d56-91c66f043f3a admin admin] PATH_INFO: `/v3/projects/7ea817fa1c264c0a9921b57c4501266d/users/0d0936c8c2a944e09109c46b129a95c8/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:07.439087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff60859-59c4-4099-8d56-91c66f043f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea817fa1c264c0a9921b57c4501266d, user_id=0d0936c8c2a944e09109c46b129a95c8, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:07.443907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff60859-59c4-4099-8d56-91c66f043f3a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:07.452764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:07.470799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 859/1701] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:07 2022] PUT /identity/v3/projects/7ea817fa1c264c0a9921b57c4501266d/users/0d0936c8c2a944e09109c46b129a95c8/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:07.487046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1814f0-7775-4adc-bb13-9c71a07d5057 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:07.487861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1814f0-7775-4adc-bb13-9c71a07d5057 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:07.489190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1814f0-7775-4adc-bb13-9c71a07d5057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:07.536637 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3b1814f0-7775-4adc-bb13-9c71a07d5057 None None] MFA Rules not processed for user `0d0936c8c2a944e09109c46b129a95c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:07.538833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3b1814f0-7775-4adc-bb13-9c71a07d5057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:07.566453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:07.795772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 843/1702] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:07 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:07.809954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff60859-59c4-4099-8d56-91c66f043f3a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:08.033241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff60859-59c4-4099-8d56-91c66f043f3a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:08.040090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351e8ac-7f77-4f0c-b772-826eaffaed38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:08.040767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e8ac-7f77-4f0c-b772-826eaffaed38 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:08.041090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e8ac-7f77-4f0c-b772-826eaffaed38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:08.041429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351e8ac-7f77-4f0c-b772-826eaffaed38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:08.090158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351e8ac-7f77-4f0c-b772-826eaffaed38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:08.098970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351e8ac-7f77-4f0c-b772-826eaffaed38 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:08.146201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 860/1703] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:07 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:08.318245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae3257-3d35-45e6-ada5-02aeb22c934e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:08.318741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae3257-3d35-45e6-ada5-02aeb22c934e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:08.319215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae3257-3d35-45e6-ada5-02aeb22c934e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:08.392683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-88ae3257-3d35-45e6-ada5-02aeb22c934e None None] MFA Rules not processed for user `b6709eed72ca46468fd298929ae0f764`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:08.395755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-88ae3257-3d35-45e6-ada5-02aeb22c934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:08.660742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 844/1704] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:08.681339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300074d1-f614-4621-9956-02abe81b218a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:08.682691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300074d1-f614-4621-9956-02abe81b218a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:08.683172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300074d1-f614-4621-9956-02abe81b218a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:08.723764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-300074d1-f614-4621-9956-02abe81b218a None None] MFA Rules not processed for user `b6709eed72ca46468fd298929ae0f764`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:08.725345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-300074d1-f614-4621-9956-02abe81b218a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:08.742938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 861/1705] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:08.753386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae3257-3d35-45e6-ada5-02aeb22c934e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:08.755303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-88ae3257-3d35-45e6-ada5-02aeb22c934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:08.932711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ae3257-3d35-45e6-ada5-02aeb22c934e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:08.946990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:08.947665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:08.947951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:08.948231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:08.971030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:08.982927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:09.008636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 845/1706] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:09.016272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:09.086290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcead2a-1158-4aac-bbe7-a63358dac5f3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:09.086775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcead2a-1158-4aac-bbe7-a63358dac5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:09.086850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcead2a-1158-4aac-bbe7-a63358dac5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:09.125068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7fcead2a-1158-4aac-bbe7-a63358dac5f3 None None] MFA Rules not processed for user `9d420e4b554e4deab21b89094e0547ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:09.126342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7fcead2a-1158-4aac-bbe7-a63358dac5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:09.178088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:09.179561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:09.289513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 862/1707] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:09.333890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f604d-873a-4f86-a396-9ee844b7131e service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:09.348175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:09.349593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:09.350772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:09.351059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] PATH_INFO: `/v3/projects/59256a4c200e481c8482bf81eee0b4bf` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:09.356586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=59256a4c200e481c8482bf81eee0b4bf)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:09.358853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:09.603847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59256a4c200e481c8482bf81eee0b4bf', 'request_id': 'req-cd5d9190-2e20-4b7d-9313-069c3d919bcb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:09.615399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59256a4c200e481c8482bf81eee0b4bf', 'request_id': 'req-cd5d9190-2e20-4b7d-9313-069c3d919bcb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:09.631205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59256a4c200e481c8482bf81eee0b4bf', 'request_id': 'req-cd5d9190-2e20-4b7d-9313-069c3d919bcb'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:09.751396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] The token cache is being invalidate because project 59256a4c200e481c8482bf81eee0b4bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:09.751836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:09.752809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cd5d9190-2e20-4b7d-9313-069c3d919bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:09.760923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 846/1708] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:09 2022] DELETE /identity/v3/projects/59256a4c200e481c8482bf81eee0b4bf => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:09.763326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcead2a-1158-4aac-bbe7-a63358dac5f3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:09.798558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcead2a-1158-4aac-bbe7-a63358dac5f3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:09.813065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:09.904833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:09.910248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:09.910756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:09.911464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] PATH_INFO: `/v3/users/bbe992d56ebc4d38a212d8b599a1e1e8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:09.917081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe992d56ebc4d38a212d8b599a1e1e8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:09.920020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:10.221516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe992d56ebc4d38a212d8b599a1e1e8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:10.250545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe992d56ebc4d38a212d8b599a1e1e8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:10.278072 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe992d56ebc4d38a212d8b599a1e1e8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:10.279963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe992d56ebc4d38a212d8b599a1e1e8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:10.360665 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c2f800c9-489e-4a62-a5a0-8d2c8155bef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe992d56ebc4d38a212d8b599a1e1e8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:10.368816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 863/1709] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:09 2022] DELETE /identity/v3/users/bbe992d56ebc4d38a212d8b599a1e1e8 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:10.774071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744fba-792a-4c6f-93a4-a6032ce57de9 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:10.775037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744fba-792a-4c6f-93a4-a6032ce57de9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:10.775494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744fba-792a-4c6f-93a4-a6032ce57de9 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:10.777504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 847/1710] 10.222.0.41 () {54 vars in 912 bytes} [Sat May 28 03:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:36:10.786419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c785d44-6260-4604-b2b0-625c3c73b544 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:10.786701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c785d44-6260-4604-b2b0-625c3c73b544 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:10.786980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c785d44-6260-4604-b2b0-625c3c73b544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:10.846511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f16036-6f15-4131-95aa-9a056e90fdc9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:10.846511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f16036-6f15-4131-95aa-9a056e90fdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:10.846511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f16036-6f15-4131-95aa-9a056e90fdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:10.876657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d5f16036-6f15-4131-95aa-9a056e90fdc9 None None] MFA Rules not processed for user `b6709eed72ca46468fd298929ae0f764`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:10.878343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d5f16036-6f15-4131-95aa-9a056e90fdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:10.880810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6c785d44-6260-4604-b2b0-625c3c73b544 None None] MFA Rules not processed for user `1dd9a0e75308460080902d1da74074c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:10.882109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6c785d44-6260-4604-b2b0-625c3c73b544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:10.981648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:11.059585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 864/1711] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:36:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:11.074975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 848/1712] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:11.218228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2d86-64c6-4629-8fa8-a49990a312ad None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:11.220223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2d86-64c6-4629-8fa8-a49990a312ad None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:11.222485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2d86-64c6-4629-8fa8-a49990a312ad None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:11.232024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 865/1713] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:36:11 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:36:11.326343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f16036-6f15-4131-95aa-9a056e90fdc9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:11.331951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d5f16036-6f15-4131-95aa-9a056e90fdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:11.349206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da45ce-9fa0-4f90-862d-5abb598d8d97 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:11.349577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da45ce-9fa0-4f90-862d-5abb598d8d97 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:11.349916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da45ce-9fa0-4f90-862d-5abb598d8d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:11.353391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f16036-6f15-4131-95aa-9a056e90fdc9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:11.374495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:11.375171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:11.376367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:11.380489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:11.413738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c3da45ce-9fa0-4f90-862d-5abb598d8d97 None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:11.415400 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c3da45ce-9fa0-4f90-862d-5abb598d8d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:11.426577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:11.433884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:11.476945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 849/1714] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:11.640387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:11.826839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 866/1715] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:36:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:13.258761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:13.263156 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:13.464926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da45ce-9fa0-4f90-862d-5abb598d8d97 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:13.467264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c3da45ce-9fa0-4f90-862d-5abb598d8d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:13.548672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359f837-6363-48e9-b36b-ca1196aec557 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:13.565362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:13.566463 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:13.566953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:13.567410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] PATH_INFO: `/v3/projects/5999c0589e3747c898aa4b030ee7f5b4` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:13.569196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5999c0589e3747c898aa4b030ee7f5b4)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:13.571558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:13.717326 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da45ce-9fa0-4f90-862d-5abb598d8d97 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:13.747270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:13.748172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:13.748499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:13.748753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] PATH_INFO: `/v3/users/8c60d488b0da4fffb7aa52d4ce900f5f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:13.754853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c60d488b0da4fffb7aa52d4ce900f5f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:13.762041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:13.783788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5999c0589e3747c898aa4b030ee7f5b4', 'request_id': 'req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:13.805619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5999c0589e3747c898aa4b030ee7f5b4', 'request_id': 'req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:13.841477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5999c0589e3747c898aa4b030ee7f5b4', 'request_id': 'req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:14.010858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] The token cache is being invalidate because project 5999c0589e3747c898aa4b030ee7f5b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:14.013387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:14.022433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76d5cfca-1d07-472a-a4ba-dc723fd29b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:14.028387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 850/1716] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:13 2022] DELETE /identity/v3/projects/5999c0589e3747c898aa4b030ee7f5b4 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:14.039128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa270c81-a9fe-4e1b-8bb5-b71413934556 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:14.039422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa270c81-a9fe-4e1b-8bb5-b71413934556 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:14.040196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa270c81-a9fe-4e1b-8bb5-b71413934556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:14.148468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-fa270c81-a9fe-4e1b-8bb5-b71413934556 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:14.160023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c60d488b0da4fffb7aa52d4ce900f5f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:14.162841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-fa270c81-a9fe-4e1b-8bb5-b71413934556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:14.191606 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c60d488b0da4fffb7aa52d4ce900f5f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:14.244956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c60d488b0da4fffb7aa52d4ce900f5f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:14.257763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c60d488b0da4fffb7aa52d4ce900f5f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:14.335641 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:14.389648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 851/1717] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:14.398613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-60495e5e-10d6-4fe2-93c2-a30fe4186a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c60d488b0da4fffb7aa52d4ce900f5f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:14.407941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 867/1718] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:13 2022] DELETE /identity/v3/users/8c60d488b0da4fffb7aa52d4ce900f5f => generated 0 bytes in 945 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:14.411448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92685d16-7dc2-469a-8b87-f3d6e6d6a32c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:14.412403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92685d16-7dc2-469a-8b87-f3d6e6d6a32c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:14.412940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92685d16-7dc2-469a-8b87-f3d6e6d6a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:14.417523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740d725-8a9d-4ae5-a3e5-2fc72687289c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:14.417818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740d725-8a9d-4ae5-a3e5-2fc72687289c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:14.418102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740d725-8a9d-4ae5-a3e5-2fc72687289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:14.454454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-92685d16-7dc2-469a-8b87-f3d6e6d6a32c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:14.457829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-92685d16-7dc2-469a-8b87-f3d6e6d6a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:14.483342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6740d725-8a9d-4ae5-a3e5-2fc72687289c None None] MFA Rules not processed for user `dc6a3335562448d1934fa18e8f51b119`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:14.488827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6740d725-8a9d-4ae5-a3e5-2fc72687289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:14.617183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:14.626591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 852/1719] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:14.628463 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed15e-9aa4-499f-8bb1-8166ff9490eb None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:14.628748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed15e-9aa4-499f-8bb1-8166ff9490eb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:14.629131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed15e-9aa4-499f-8bb1-8166ff9490eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:14.660398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f7bed15e-9aa4-499f-8bb1-8166ff9490eb None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:14.661876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f7bed15e-9aa4-499f-8bb1-8166ff9490eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:14.675947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 853/1720] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:14.681522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 868/1721] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:14.684422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bed15e-9aa4-499f-8bb1-8166ff9490eb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:14.710826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92685d16-7dc2-469a-8b87-f3d6e6d6a32c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:14.712687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-92685d16-7dc2-469a-8b87-f3d6e6d6a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:14.740157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bed15e-9aa4-499f-8bb1-8166ff9490eb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:14.760334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:14.761824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:14.763489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:14.764043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:14.767939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:14.768591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:14.772092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:14.814516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 854/1722] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:36:14.820403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:14.862496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d79fe-cbd4-40c6-b3a5-b6ec9f56ea07 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:14.880121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58729e2-9322-4f9d-8aa8-6e56436966c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:14.886575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58729e2-9322-4f9d-8aa8-6e56436966c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:14.886575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58729e2-9322-4f9d-8aa8-6e56436966c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:14.886575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58729e2-9322-4f9d-8aa8-6e56436966c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:14.891924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58729e2-9322-4f9d-8aa8-6e56436966c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:14.894060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58729e2-9322-4f9d-8aa8-6e56436966c7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:14.989346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92685d16-7dc2-469a-8b87-f3d6e6d6a32c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:15.001534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:15.003134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:15.006433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:15.006963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:15.020815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:15.022571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:15.050504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 869/1723] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:15.071490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:15.073660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:15.093073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 855/1724] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:14 2022] POST /identity/v3/projects => generated 357 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:15.098118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58729e2-9322-4f9d-8aa8-6e56436966c7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:15.369035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0563-bcb5-4fed-b43c-8e094e3811b6 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:15.435963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58729e2-9322-4f9d-8aa8-6e56436966c7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:15.463349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae7e7a-3fc7-4a88-b2dd-b8365e6c1373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:15.466070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae7e7a-3fc7-4a88-b2dd-b8365e6c1373 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:15.466590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae7e7a-3fc7-4a88-b2dd-b8365e6c1373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:15.467022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae7e7a-3fc7-4a88-b2dd-b8365e6c1373 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:15.468610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae7e7a-3fc7-4a88-b2dd-b8365e6c1373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:15.473901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae7e7a-3fc7-4a88-b2dd-b8365e6c1373 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:15.509050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:15.511706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:15.512842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:15.513931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:15.520572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:15.564794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:15.564794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:15.588071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 870/1725] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:15.660248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 856/1726] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:15 2022] POST /identity/v3/users => generated 279 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:15.664064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:15.686472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc5092-9a22-49ba-b331-4cb508a42bbc service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:15.703727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e44e5-e8dc-4c33-bef9-23424a053a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:15.704795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e44e5-e8dc-4c33-bef9-23424a053a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:15.705154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e44e5-e8dc-4c33-bef9-23424a053a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:15.705506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e44e5-e8dc-4c33-bef9-23424a053a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:15.706895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e44e5-e8dc-4c33-bef9-23424a053a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:15.708638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e44e5-e8dc-4c33-bef9-23424a053a99 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:15.762705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 871/1727] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:15.765548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae7e7a-3fc7-4a88-b2dd-b8365e6c1373 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:15.785458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae7e7a-3fc7-4a88-b2dd-b8365e6c1373 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:15.795897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695364e8-d5e8-4897-998b-13dffa6933e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:15.796591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695364e8-d5e8-4897-998b-13dffa6933e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:15.796879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695364e8-d5e8-4897-998b-13dffa6933e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:15.797188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695364e8-d5e8-4897-998b-13dffa6933e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:15.798040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695364e8-d5e8-4897-998b-13dffa6933e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:15.808030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695364e8-d5e8-4897-998b-13dffa6933e2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:15.882750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 857/1728] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:15.883739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e44e5-e8dc-4c33-bef9-23424a053a99 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:15.900917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e44e5-e8dc-4c33-bef9-23424a053a99 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:15.930194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7b432-4755-4bd2-9e70-aeca5a65f212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:15.931037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7b432-4755-4bd2-9e70-aeca5a65f212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:15.931325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7b432-4755-4bd2-9e70-aeca5a65f212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:15.931637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7b432-4755-4bd2-9e70-aeca5a65f212 admin admin] PATH_INFO: `/v3/projects/8fd94d5e12cc4aefa06f07f7884ce865/users/7a6af81a87d8464999fa02d8a5d20f6a/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:15.965349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7b432-4755-4bd2-9e70-aeca5a65f212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd94d5e12cc4aefa06f07f7884ce865, user_id=7a6af81a87d8464999fa02d8a5d20f6a, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:15.968751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7b432-4755-4bd2-9e70-aeca5a65f212 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:16.093314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d49f8-2b20-46cb-8baf-ce2d9714eabf None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:16.093618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d49f8-2b20-46cb-8baf-ce2d9714eabf None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:16.093900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d49f8-2b20-46cb-8baf-ce2d9714eabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:16.095858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 872/1729] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:15 2022] PUT /identity/v3/projects/8fd94d5e12cc4aefa06f07f7884ce865/users/7a6af81a87d8464999fa02d8a5d20f6a/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:16.128869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f54d49f8-2b20-46cb-8baf-ce2d9714eabf None None] MFA Rules not processed for user `7a6af81a87d8464999fa02d8a5d20f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:16.130175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f54d49f8-2b20-46cb-8baf-ce2d9714eabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:16.272662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7b432-4755-4bd2-9e70-aeca5a65f212 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:16.283433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 858/1730] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:16 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:16.419833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7b432-4755-4bd2-9e70-aeca5a65f212 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:16.430842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4cf9-c88a-4cf3-ad50-b7b2c2764fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:16.431554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4cf9-c88a-4cf3-ad50-b7b2c2764fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:16.431844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4cf9-c88a-4cf3-ad50-b7b2c2764fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:16.437640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4cf9-c88a-4cf3-ad50-b7b2c2764fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:16.437640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e4cf9-c88a-4cf3-ad50-b7b2c2764fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:16.437640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e4cf9-c88a-4cf3-ad50-b7b2c2764fd1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:16.459601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:16.583381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d49f8-2b20-46cb-8baf-ce2d9714eabf None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:16.589765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 873/1731] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:16 2022] POST /identity/v3/projects => generated 355 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:16.604428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d49f8-2b20-46cb-8baf-ce2d9714eabf None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:16.741452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:16.766429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7023a-08e6-4a4d-9b55-d85ad1027e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:16.767199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7023a-08e6-4a4d-9b55-d85ad1027e59 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:16.767491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7023a-08e6-4a4d-9b55-d85ad1027e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:16.767773 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7023a-08e6-4a4d-9b55-d85ad1027e59 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:16.770743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7023a-08e6-4a4d-9b55-d85ad1027e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:16.782547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7023a-08e6-4a4d-9b55-d85ad1027e59 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:16.908542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 859/1732] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:16 2022] POST /identity/v3/users => generated 284 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:16.916829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4cf9-c88a-4cf3-ad50-b7b2c2764fd1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:16.934085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4cf9-c88a-4cf3-ad50-b7b2c2764fd1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:16.939282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa57e-7290-4f63-9154-499d666bf5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:16.940012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa57e-7290-4f63-9154-499d666bf5be admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:16.940861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa57e-7290-4f63-9154-499d666bf5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:16.941180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa57e-7290-4f63-9154-499d666bf5be admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:16.941932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fa57e-7290-4f63-9154-499d666bf5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:16.943621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fa57e-7290-4f63-9154-499d666bf5be admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:16.962231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 874/1733] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:16.963363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7023a-08e6-4a4d-9b55-d85ad1027e59 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:16.984213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7023a-08e6-4a4d-9b55-d85ad1027e59 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:17.006013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78fc9a-9881-472f-9e8c-c610590dab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:17.013517 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78fc9a-9881-472f-9e8c-c610590dab9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:17.013517 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78fc9a-9881-472f-9e8c-c610590dab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:17.013517 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78fc9a-9881-472f-9e8c-c610590dab9b admin admin] PATH_INFO: `/v3/projects/c2f33339c845430a8bc86b5a56cba1c9/users/c2ef777b15fc4b16ae6ddb1f4784dc1b/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:17.039521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78fc9a-9881-472f-9e8c-c610590dab9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f33339c845430a8bc86b5a56cba1c9, user_id=c2ef777b15fc4b16ae6ddb1f4784dc1b, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:17.042565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78fc9a-9881-472f-9e8c-c610590dab9b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:17.132076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 860/1734] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:16 2022] PUT /identity/v3/projects/c2f33339c845430a8bc86b5a56cba1c9/users/c2ef777b15fc4b16ae6ddb1f4784dc1b/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:17.135998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa57e-7290-4f63-9154-499d666bf5be admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:17.151512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa57e-7290-4f63-9154-499d666bf5be admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:17.226111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a56fd-8af4-4e6c-b0f1-8427ae7fdd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:17.227050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a56fd-8af4-4e6c-b0f1-8427ae7fdd6e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:17.227299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a56fd-8af4-4e6c-b0f1-8427ae7fdd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:17.227845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a56fd-8af4-4e6c-b0f1-8427ae7fdd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:17.228691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a56fd-8af4-4e6c-b0f1-8427ae7fdd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:17.230457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a56fd-8af4-4e6c-b0f1-8427ae7fdd6e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:17.259484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 875/1735] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:17.261614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78fc9a-9881-472f-9e8c-c610590dab9b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:17.281877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78fc9a-9881-472f-9e8c-c610590dab9b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:17.296512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:17.297603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:17.297958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:17.298306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] PATH_INFO: `/v3/projects/c2f33339c845430a8bc86b5a56cba1c9/users/c2ef777b15fc4b16ae6ddb1f4784dc1b/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:17.302220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f33339c845430a8bc86b5a56cba1c9, user_id=c2ef777b15fc4b16ae6ddb1f4784dc1b, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:17.305043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:17.318782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 861/1736] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:17 2022] PUT /identity/v3/projects/c2f33339c845430a8bc86b5a56cba1c9/users/c2ef777b15fc4b16ae6ddb1f4784dc1b/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:17.330178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696d6f-6562-4a42-bc76-b1999caee031 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:17.330557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696d6f-6562-4a42-bc76-b1999caee031 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:17.330904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696d6f-6562-4a42-bc76-b1999caee031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:17.369013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-94696d6f-6562-4a42-bc76-b1999caee031 None None] MFA Rules not processed for user `c2ef777b15fc4b16ae6ddb1f4784dc1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:17.370968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-94696d6f-6562-4a42-bc76-b1999caee031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:17.430133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:17.432895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:17.494965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 876/1737] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:17 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:17.514202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:17.518230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696d6f-6562-4a42-bc76-b1999caee031 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:17.524108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-94696d6f-6562-4a42-bc76-b1999caee031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:17.595712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de90dde-b775-4c12-ab4c-7ce59229ca2e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:17.612959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:17.614379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:17.614680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:17.614959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] PATH_INFO: `/v3/projects/47d1882059f848cbaf38514295d3fcdd` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:17.617791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d1882059f848cbaf38514295d3fcdd)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:17.619767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:17.623489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696d6f-6562-4a42-bc76-b1999caee031 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:17.635446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d537cf1-b70b-4d6d-b294-28d150d2a39e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:17.638567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d537cf1-b70b-4d6d-b294-28d150d2a39e service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:17.639346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d537cf1-b70b-4d6d-b294-28d150d2a39e service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:17.639848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d537cf1-b70b-4d6d-b294-28d150d2a39e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:17.653537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d537cf1-b70b-4d6d-b294-28d150d2a39e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:17.655380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d537cf1-b70b-4d6d-b294-28d150d2a39e service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:17.673264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 877/1738] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:17 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:17.701996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:17.844003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d1882059f848cbaf38514295d3fcdd', 'request_id': 'req-734885da-0c2b-4b11-88ad-d1f32ef729c4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:17.851832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d1882059f848cbaf38514295d3fcdd', 'request_id': 'req-734885da-0c2b-4b11-88ad-d1f32ef729c4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:17.859163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d1882059f848cbaf38514295d3fcdd', 'request_id': 'req-734885da-0c2b-4b11-88ad-d1f32ef729c4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:17.933913 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] The token cache is being invalidate because project 47d1882059f848cbaf38514295d3fcdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:17.940005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:17.940005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:17.947615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 862/1739] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:17 2022] DELETE /identity/v3/projects/47d1882059f848cbaf38514295d3fcdd => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:17.950045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d537cf1-b70b-4d6d-b294-28d150d2a39e service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:17.957378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:17.960508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:17.970585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d537cf1-b70b-4d6d-b294-28d150d2a39e service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:18.263503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:18.277959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734885da-0c2b-4b11-88ad-d1f32ef729c4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:18.281409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:18.281409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:18.281697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] PATH_INFO: `/v3/users/0d0936c8c2a944e09109c46b129a95c8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:18.289849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ba29a-869b-4f4b-93d9-ef9fbe7d51e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:18.290906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ba29a-869b-4f4b-93d9-ef9fbe7d51e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:18.291433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ba29a-869b-4f4b-93d9-ef9fbe7d51e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:18.292124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0936c8c2a944e09109c46b129a95c8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:18.292612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ba29a-869b-4f4b-93d9-ef9fbe7d51e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:18.294177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:18.297052 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2e9ba29a-869b-4f4b-93d9-ef9fbe7d51e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:18.438265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ba29a-869b-4f4b-93d9-ef9fbe7d51e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:18.441080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ba29a-869b-4f4b-93d9-ef9fbe7d51e4 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:18.459055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 863/1740] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:17 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:18.473220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c64a0-ea07-492c-8df1-d65a6d008524 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:18.475305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c64a0-ea07-492c-8df1-d65a6d008524 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:18.475756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c64a0-ea07-492c-8df1-d65a6d008524 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:18.482602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 864/1741] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:36:18 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:36:18.611118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2fcb-eb97-43c6-bb07-00370dd9ed4c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:18.613563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2fcb-eb97-43c6-bb07-00370dd9ed4c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:18.614072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa2fcb-eb97-43c6-bb07-00370dd9ed4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:18.656437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-68aa2fcb-eb97-43c6-bb07-00370dd9ed4c None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:18.661241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-68aa2fcb-eb97-43c6-bb07-00370dd9ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:18.668389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0936c8c2a944e09109c46b129a95c8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:18.877413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0936c8c2a944e09109c46b129a95c8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:18.877413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0936c8c2a944e09109c46b129a95c8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:18.877413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0936c8c2a944e09109c46b129a95c8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:18.878935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 865/1742] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:36:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:19.267316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0936c8c2a944e09109c46b129a95c8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:19.267316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 878/1743] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:17 2022] DELETE /identity/v3/users/0d0936c8c2a944e09109c46b129a95c8 => generated 0 bytes in 1299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:19.403960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa2fcb-eb97-43c6-bb07-00370dd9ed4c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:19.405912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-68aa2fcb-eb97-43c6-bb07-00370dd9ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:19.428931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa2fcb-eb97-43c6-bb07-00370dd9ed4c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:19.595056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf3ade-7c64-49a0-895d-3ac028478ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:19.595727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf3ade-7c64-49a0-895d-3ac028478ca8 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:19.596175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf3ade-7c64-49a0-895d-3ac028478ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:19.597012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf3ade-7c64-49a0-895d-3ac028478ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:19.602771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7fcf3ade-7c64-49a0-895d-3ac028478ca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:19.622603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf3ade-7c64-49a0-895d-3ac028478ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:19.634110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf3ade-7c64-49a0-895d-3ac028478ca8 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:19.673169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 866/1744] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:20.128749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:20.759749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:20.764781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:20.780682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:20.861146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2ff16-b1fb-40bb-b11f-a03a05c3fda7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:20.866778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:20.867643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:20.867929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:20.868198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] PATH_INFO: `/v3/projects/7ea817fa1c264c0a9921b57c4501266d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:20.869396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea817fa1c264c0a9921b57c4501266d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:20.871509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:21.033048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea817fa1c264c0a9921b57c4501266d', 'request_id': 'req-33becb2e-bed6-4602-b334-6826bbd11b1c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:21.039876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea817fa1c264c0a9921b57c4501266d', 'request_id': 'req-33becb2e-bed6-4602-b334-6826bbd11b1c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:21.045736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea817fa1c264c0a9921b57c4501266d', 'request_id': 'req-33becb2e-bed6-4602-b334-6826bbd11b1c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:21.134551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] The token cache is being invalidate because project 7ea817fa1c264c0a9921b57c4501266d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:21.135815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:21.135815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33becb2e-bed6-4602-b334-6826bbd11b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:21.137013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 879/1745] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:20 2022] DELETE /identity/v3/projects/7ea817fa1c264c0a9921b57c4501266d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:21.142013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce392c5-ee39-499f-93b1-4136ce3ed13c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.142472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce392c5-ee39-499f-93b1-4136ce3ed13c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.142566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce392c5-ee39-499f-93b1-4136ce3ed13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.161046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2ce392c5-ee39-499f-93b1-4136ce3ed13c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:21.162261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2ce392c5-ee39-499f-93b1-4136ce3ed13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:21.229937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 867/1746] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:21.239041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ad560-8bba-4c7c-a74d-ae81d9e0bd47 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.239402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ad560-8bba-4c7c-a74d-ae81d9e0bd47 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.241904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ad560-8bba-4c7c-a74d-ae81d9e0bd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.258225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-714ad560-8bba-4c7c-a74d-ae81d9e0bd47 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:21.259431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-714ad560-8bba-4c7c-a74d-ae81d9e0bd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:21.267357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 880/1747] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:21.272105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce392c5-ee39-499f-93b1-4136ce3ed13c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:21.281766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce392c5-ee39-499f-93b1-4136ce3ed13c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:21.287713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:21.288370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.288654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.288929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.289703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:21.290013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:21.291631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:21.303648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 868/1748] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:36:21.308119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ad560-8bba-4c7c-a74d-ae81d9e0bd47 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:21.317353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ad560-8bba-4c7c-a74d-ae81d9e0bd47 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:21.322073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493ed5e-126b-40d3-b95e-7085592fec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:21.322748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493ed5e-126b-40d3-b95e-7085592fec8c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.323034 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493ed5e-126b-40d3-b95e-7085592fec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.323370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493ed5e-126b-40d3-b95e-7085592fec8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.324086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493ed5e-126b-40d3-b95e-7085592fec8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:21.326047 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493ed5e-126b-40d3-b95e-7085592fec8c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:21.412906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 881/1749] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:21 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:21.416454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:21.426667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054aa34-f711-4df7-82f0-0182a7f8fb9a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:21.487567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c64fbd-e5d4-4f45-af93-7ac7c95f4099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:21.493939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c64fbd-e5d4-4f45-af93-7ac7c95f4099 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.493939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c64fbd-e5d4-4f45-af93-7ac7c95f4099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.493939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c64fbd-e5d4-4f45-af93-7ac7c95f4099 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.493939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c64fbd-e5d4-4f45-af93-7ac7c95f4099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:21.493939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c64fbd-e5d4-4f45-af93-7ac7c95f4099 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:21.689703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 869/1750] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:21 2022] POST /identity/v3/users => generated 294 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:21.695707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493ed5e-126b-40d3-b95e-7085592fec8c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:21.706827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493ed5e-126b-40d3-b95e-7085592fec8c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:21.714178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c1a66-2283-42c6-8c97-2bdd24c05483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:21.715010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c1a66-2283-42c6-8c97-2bdd24c05483 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.715387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c1a66-2283-42c6-8c97-2bdd24c05483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.715773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c1a66-2283-42c6-8c97-2bdd24c05483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.716592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c1a66-2283-42c6-8c97-2bdd24c05483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:21.718339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c1a66-2283-42c6-8c97-2bdd24c05483 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:21.731346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 882/1751] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:21.735230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c64fbd-e5d4-4f45-af93-7ac7c95f4099 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:21.750183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c64fbd-e5d4-4f45-af93-7ac7c95f4099 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:21.756498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81c78c-cfc9-4f31-b9bb-419458ab478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:21.759440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81c78c-cfc9-4f31-b9bb-419458ab478a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.759440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81c78c-cfc9-4f31-b9bb-419458ab478a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.759440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81c78c-cfc9-4f31-b9bb-419458ab478a admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.759440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81c78c-cfc9-4f31-b9bb-419458ab478a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:21.761144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81c78c-cfc9-4f31-b9bb-419458ab478a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:21.774587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 870/1752] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:21.779589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c1a66-2283-42c6-8c97-2bdd24c05483 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:21.793836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c1a66-2283-42c6-8c97-2bdd24c05483 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:21.798595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f9fd4-04a3-4472-97b5-96b713643faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:21.799703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9fd4-04a3-4472-97b5-96b713643faa admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.799985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9fd4-04a3-4472-97b5-96b713643faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.800266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9fd4-04a3-4472-97b5-96b713643faa admin admin] PATH_INFO: `/v3/projects/ceff16afd90a4f1ca3c59b937ca20836/users/ed2bb6dcdb1e43d7b4641fb5b1376286/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.823972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f9fd4-04a3-4472-97b5-96b713643faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceff16afd90a4f1ca3c59b937ca20836, user_id=ed2bb6dcdb1e43d7b4641fb5b1376286, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:21.827996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f9fd4-04a3-4472-97b5-96b713643faa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:21.950901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 883/1753] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:21 2022] PUT /identity/v3/projects/ceff16afd90a4f1ca3c59b937ca20836/users/ed2bb6dcdb1e43d7b4641fb5b1376286/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:21.960770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f3536-f3ed-4514-be64-c9023f7a28fc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:21.961148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f3536-f3ed-4514-be64-c9023f7a28fc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:21.961523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f3536-f3ed-4514-be64-c9023f7a28fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:21.999554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-347f3536-f3ed-4514-be64-c9023f7a28fc None None] MFA Rules not processed for user `ed2bb6dcdb1e43d7b4641fb5b1376286`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:22.000886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-347f3536-f3ed-4514-be64-c9023f7a28fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:22.085564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 871/1754] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:22.090270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f9fd4-04a3-4472-97b5-96b713643faa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:22.158083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f9fd4-04a3-4472-97b5-96b713643faa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:22.162876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c3f69-c2ea-46da-a387-680b38f06ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:22.163536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c3f69-c2ea-46da-a387-680b38f06ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:22.163822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c3f69-c2ea-46da-a387-680b38f06ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:22.164097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c3f69-c2ea-46da-a387-680b38f06ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:22.164776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c3f69-c2ea-46da-a387-680b38f06ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:22.166799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c3f69-c2ea-46da-a387-680b38f06ab4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:22.199551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:22.257266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 884/1755] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:22 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:22.261338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f3536-f3ed-4514-be64-c9023f7a28fc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:22.272138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f3536-f3ed-4514-be64-c9023f7a28fc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:22.339731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4f1b-7adc-4143-a028-b7acc5984b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:22.340482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4f1b-7adc-4143-a028-b7acc5984b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:22.340757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4f1b-7adc-4143-a028-b7acc5984b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:22.341037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4f1b-7adc-4143-a028-b7acc5984b57 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:22.347014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b4f1b-7adc-4143-a028-b7acc5984b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:22.347014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b4f1b-7adc-4143-a028-b7acc5984b57 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:22.411018 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 872/1756] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:22 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:22.415230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c3f69-c2ea-46da-a387-680b38f06ab4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:22.426595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c3f69-c2ea-46da-a387-680b38f06ab4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:22.431634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b1a09-6eb8-45c0-b132-7b5fc04780d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:22.432286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b1a09-6eb8-45c0-b132-7b5fc04780d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:22.432569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b1a09-6eb8-45c0-b132-7b5fc04780d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:22.432846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b1a09-6eb8-45c0-b132-7b5fc04780d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:22.433603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b1a09-6eb8-45c0-b132-7b5fc04780d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:22.435208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b1a09-6eb8-45c0-b132-7b5fc04780d3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:22.446247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 885/1757] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:22.450481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4f1b-7adc-4143-a028-b7acc5984b57 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:22.460757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4f1b-7adc-4143-a028-b7acc5984b57 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:22.467349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152b630-4233-42f0-8b70-7c64a9c63c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:22.467349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152b630-4233-42f0-8b70-7c64a9c63c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:22.467349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152b630-4233-42f0-8b70-7c64a9c63c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:22.467349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152b630-4233-42f0-8b70-7c64a9c63c2c admin admin] PATH_INFO: `/v3/projects/c02f1e520e824d8788a3014f743a6996/users/a7c2ec7a57cd4f508ac12bdf28191b19/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:22.482359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152b630-4233-42f0-8b70-7c64a9c63c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02f1e520e824d8788a3014f743a6996, user_id=a7c2ec7a57cd4f508ac12bdf28191b19, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:22.483952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:22.485128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152b630-4233-42f0-8b70-7c64a9c63c2c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:22.591936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 873/1758] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:22 2022] PUT /identity/v3/projects/c02f1e520e824d8788a3014f743a6996/users/a7c2ec7a57cd4f508ac12bdf28191b19/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:22.595810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b1a09-6eb8-45c0-b132-7b5fc04780d3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:22.606303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b1a09-6eb8-45c0-b132-7b5fc04780d3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:22.662789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de071a-e8d3-422d-9622-1c45669f627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:22.663452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de071a-e8d3-422d-9622-1c45669f627f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:22.663737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de071a-e8d3-422d-9622-1c45669f627f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:22.664021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de071a-e8d3-422d-9622-1c45669f627f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:22.664905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de071a-e8d3-422d-9622-1c45669f627f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:22.666535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de071a-e8d3-422d-9622-1c45669f627f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:22.676129 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 886/1759] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:22.680498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152b630-4233-42f0-8b70-7c64a9c63c2c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:22.690660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152b630-4233-42f0-8b70-7c64a9c63c2c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:22.695253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:22.696118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:22.696397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:22.696672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] PATH_INFO: `/v3/projects/c02f1e520e824d8788a3014f743a6996/users/a7c2ec7a57cd4f508ac12bdf28191b19/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:22.698642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02f1e520e824d8788a3014f743a6996, user_id=a7c2ec7a57cd4f508ac12bdf28191b19, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:22.701293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:22.712058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 874/1760] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:22 2022] PUT /identity/v3/projects/c02f1e520e824d8788a3014f743a6996/users/a7c2ec7a57cd4f508ac12bdf28191b19/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:22.723450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868251d-2ef9-4a11-98f6-2bc7b4786900 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:22.723749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868251d-2ef9-4a11-98f6-2bc7b4786900 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:22.724036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868251d-2ef9-4a11-98f6-2bc7b4786900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:22.882297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e868251d-2ef9-4a11-98f6-2bc7b4786900 None None] MFA Rules not processed for user `a7c2ec7a57cd4f508ac12bdf28191b19`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:22.883553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e868251d-2ef9-4a11-98f6-2bc7b4786900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:22.968153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 887/1761] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:22.980669 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:22.982379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:23.047600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:23.082995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749aa181-3223-4807-a8c0-7a8146c9a3aa admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:23.090722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04157d15-0de9-41c8-ba5f-df174e52eb39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:23.091520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04157d15-0de9-41c8-ba5f-df174e52eb39 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:23.091910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04157d15-0de9-41c8-ba5f-df174e52eb39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:23.093795 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04157d15-0de9-41c8-ba5f-df174e52eb39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:23.104672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04157d15-0de9-41c8-ba5f-df174e52eb39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:23.112412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04157d15-0de9-41c8-ba5f-df174e52eb39 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:23.124311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 875/1762] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:23.247325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868251d-2ef9-4a11-98f6-2bc7b4786900 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:23.290269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:23.323174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868251d-2ef9-4a11-98f6-2bc7b4786900 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:23.327702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:23.328574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:23.328902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:23.329219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] PATH_INFO: `/v3/users/ed2bb6dcdb1e43d7b4641fb5b1376286` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:23.330396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2bb6dcdb1e43d7b4641fb5b1376286)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:23.332893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:23.583145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2bb6dcdb1e43d7b4641fb5b1376286'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:23.590883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2bb6dcdb1e43d7b4641fb5b1376286'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:23.602845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2bb6dcdb1e43d7b4641fb5b1376286'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:23.603562 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2bb6dcdb1e43d7b4641fb5b1376286'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:23.673460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2bb6dcdb1e43d7b4641fb5b1376286'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:23.680220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 888/1763] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:23 2022] DELETE /identity/v3/users/ed2bb6dcdb1e43d7b4641fb5b1376286 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:23.684757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723be0a-6ef9-4e90-a8e9-628b104f26da None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:23.685253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723be0a-6ef9-4e90-a8e9-628b104f26da None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:23.685476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3723be0a-6ef9-4e90-a8e9-628b104f26da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:23.703546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3723be0a-6ef9-4e90-a8e9-628b104f26da None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:23.704788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3723be0a-6ef9-4e90-a8e9-628b104f26da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:23.761904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 876/1764] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:23.771679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:23.773395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:23.832978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5004fb-8482-49ac-a425-dec420baaa89 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:23.836499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c473740-c480-4581-bf3d-450bbd1a2722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:23.837166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c473740-c480-4581-bf3d-450bbd1a2722 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:23.837451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c473740-c480-4581-bf3d-450bbd1a2722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:23.837729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c473740-c480-4581-bf3d-450bbd1a2722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:23.847200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c473740-c480-4581-bf3d-450bbd1a2722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:23.848622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c473740-c480-4581-bf3d-450bbd1a2722 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:23.858704 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 889/1765] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:24.409407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723be0a-6ef9-4e90-a8e9-628b104f26da None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:24.411212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3723be0a-6ef9-4e90-a8e9-628b104f26da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:24.427345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3723be0a-6ef9-4e90-a8e9-628b104f26da None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:24.446332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:24.447887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:24.448382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:24.448971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] PATH_INFO: `/v3/projects/ceff16afd90a4f1ca3c59b937ca20836` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:24.454942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceff16afd90a4f1ca3c59b937ca20836)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:24.456966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:24.595828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceff16afd90a4f1ca3c59b937ca20836', 'request_id': 'req-30e2acbe-8e72-4931-83cb-6789291c36e7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:24.602402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceff16afd90a4f1ca3c59b937ca20836', 'request_id': 'req-30e2acbe-8e72-4931-83cb-6789291c36e7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:24.608489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceff16afd90a4f1ca3c59b937ca20836', 'request_id': 'req-30e2acbe-8e72-4931-83cb-6789291c36e7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:24.684454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] The token cache is being invalidate because project ceff16afd90a4f1ca3c59b937ca20836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:24.684745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:24.685400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:24.691427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 877/1766] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:24 2022] DELETE /identity/v3/projects/ceff16afd90a4f1ca3c59b937ca20836 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:24.692497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c473740-c480-4581-bf3d-450bbd1a2722 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:24.702707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c473740-c480-4581-bf3d-450bbd1a2722 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:24.748101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:24.759849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:24.760773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:24.761058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:24.761399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] PATH_INFO: `/v3/users/a7c2ec7a57cd4f508ac12bdf28191b19` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:24.762596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7c2ec7a57cd4f508ac12bdf28191b19)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:24.764581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:25.002064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7c2ec7a57cd4f508ac12bdf28191b19'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:25.010496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7c2ec7a57cd4f508ac12bdf28191b19'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:25.023604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7c2ec7a57cd4f508ac12bdf28191b19'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:25.024390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7c2ec7a57cd4f508ac12bdf28191b19'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:25.107736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0ed2e0f9-8384-4fc7-96d0-3528a0170191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7c2ec7a57cd4f508ac12bdf28191b19'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:25.115169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 890/1767] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:24 2022] DELETE /identity/v3/users/a7c2ec7a57cd4f508ac12bdf28191b19 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:26.960382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:26.962551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:26.989607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2acbe-8e72-4931-83cb-6789291c36e7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:27.119956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:27.131459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:27.132667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.133200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.133659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] PATH_INFO: `/v3/projects/c02f1e520e824d8788a3014f743a6996` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.135205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02f1e520e824d8788a3014f743a6996)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:27.137661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:27.192395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93901-b077-4be2-b0ba-a8b98b77310a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.192833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93901-b077-4be2-b0ba-a8b98b77310a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.192968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93901-b077-4be2-b0ba-a8b98b77310a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.195189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 891/1768] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:36:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:36:27.259527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af308c3c-a923-4aaf-ab0c-1689ab92054b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.260011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af308c3c-a923-4aaf-ab0c-1689ab92054b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.260442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af308c3c-a923-4aaf-ab0c-1689ab92054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.291180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02f1e520e824d8788a3014f743a6996', 'request_id': 'req-020f008b-8f98-4c0b-a2d0-f8c707f4077c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:27.297466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-af308c3c-a923-4aaf-ab0c-1689ab92054b None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:27.298701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-af308c3c-a923-4aaf-ab0c-1689ab92054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:27.302927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02f1e520e824d8788a3014f743a6996', 'request_id': 'req-020f008b-8f98-4c0b-a2d0-f8c707f4077c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:27.310478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02f1e520e824d8788a3014f743a6996', 'request_id': 'req-020f008b-8f98-4c0b-a2d0-f8c707f4077c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:27.403802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] The token cache is being invalidate because project c02f1e520e824d8788a3014f743a6996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:27.405392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:27.405392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-020f008b-8f98-4c0b-a2d0-f8c707f4077c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:27.407513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 878/1769] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:26 2022] DELETE /identity/v3/projects/c02f1e520e824d8788a3014f743a6996 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:27.412035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0001d1d-9625-4077-9aec-b7905945b833 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.412332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0001d1d-9625-4077-9aec-b7905945b833 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.412619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0001d1d-9625-4077-9aec-b7905945b833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.429841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 892/1770] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:27.440876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e0001d1d-9625-4077-9aec-b7905945b833 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:27.440876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e0001d1d-9625-4077-9aec-b7905945b833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:27.573408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 879/1771] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:27.578247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf7cc0-cd03-498b-b9f7-c44c21f5f12e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.578981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf7cc0-cd03-498b-b9f7-c44c21f5f12e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.579278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf7cc0-cd03-498b-b9f7-c44c21f5f12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.599709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0001d1d-9625-4077-9aec-b7905945b833 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:27.605685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e0001d1d-9625-4077-9aec-b7905945b833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:27.624609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c6cf7cc0-cd03-498b-b9f7-c44c21f5f12e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:27.625859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c6cf7cc0-cd03-498b-b9f7-c44c21f5f12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:27.635252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0001d1d-9625-4077-9aec-b7905945b833 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:27.649682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 893/1772] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:27.652646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf7cc0-cd03-498b-b9f7-c44c21f5f12e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:27.660996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b38b5c-0ccd-430c-acaa-88bc4dc0508e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:27.662701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b38b5c-0ccd-430c-acaa-88bc4dc0508e service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.663000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b38b5c-0ccd-430c-acaa-88bc4dc0508e service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.663287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b38b5c-0ccd-430c-acaa-88bc4dc0508e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.677406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf7cc0-cd03-498b-b9f7-c44c21f5f12e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:27.680984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b38b5c-0ccd-430c-acaa-88bc4dc0508e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:27.682656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b38b5c-0ccd-430c-acaa-88bc4dc0508e service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:27.692398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:27.693074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.693398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.693684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.694427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:27.694741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:27.697365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 880/1773] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:27.698589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:27.717208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b38b5c-0ccd-430c-acaa-88bc4dc0508e service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:27.719145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 894/1774] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:36:27.729459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b38b5c-0ccd-430c-acaa-88bc4dc0508e service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:27.734823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97e58-d8c9-4a83-ae3e-8721e9952188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:27.735788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97e58-d8c9-4a83-ae3e-8721e9952188 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.736084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97e58-d8c9-4a83-ae3e-8721e9952188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.736631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e97e58-d8c9-4a83-ae3e-8721e9952188 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.737482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e97e58-d8c9-4a83-ae3e-8721e9952188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:27.739619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e97e58-d8c9-4a83-ae3e-8721e9952188 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:27.842908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:27.852666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 881/1775] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:27 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:27.864215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8d5f1-e9df-420d-b2e4-c017040967c9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:27.913163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:27.926317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83e350-190a-4f1c-b083-f511bf917846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:27.926993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83e350-190a-4f1c-b083-f511bf917846 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:27.927331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83e350-190a-4f1c-b083-f511bf917846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:27.927614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83e350-190a-4f1c-b083-f511bf917846 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:27.928321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83e350-190a-4f1c-b083-f511bf917846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:27.930252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83e350-190a-4f1c-b083-f511bf917846 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:28.051057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 895/1776] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:27 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:28.057171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97e58-d8c9-4a83-ae3e-8721e9952188 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:28.077765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e97e58-d8c9-4a83-ae3e-8721e9952188 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:28.085744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf0006-bae2-4eab-bc70-288d12838161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:28.087309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0006-bae2-4eab-bc70-288d12838161 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:28.088227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0006-bae2-4eab-bc70-288d12838161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:28.089440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf0006-bae2-4eab-bc70-288d12838161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:28.090449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf0006-bae2-4eab-bc70-288d12838161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:28.092776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf0006-bae2-4eab-bc70-288d12838161 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:28.107168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 882/1777] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:28.112520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83e350-190a-4f1c-b083-f511bf917846 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:28.124384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83e350-190a-4f1c-b083-f511bf917846 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:28.131429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a323c-711a-4744-9f10-e7caa3dbd92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:28.132237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a323c-711a-4744-9f10-e7caa3dbd92e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:28.132595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a323c-711a-4744-9f10-e7caa3dbd92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:28.132952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a323c-711a-4744-9f10-e7caa3dbd92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:28.133799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a323c-711a-4744-9f10-e7caa3dbd92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:28.135773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a323c-711a-4744-9f10-e7caa3dbd92e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:28.150022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 896/1778] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:28.154845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf0006-bae2-4eab-bc70-288d12838161 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:28.172170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf0006-bae2-4eab-bc70-288d12838161 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:28.193563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1573b0c-e034-4e50-965f-7f07de6c10fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:28.194793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573b0c-e034-4e50-965f-7f07de6c10fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:28.195358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573b0c-e034-4e50-965f-7f07de6c10fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:28.195959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573b0c-e034-4e50-965f-7f07de6c10fd admin admin] PATH_INFO: `/v3/projects/66554ec870b7428ca6a1226686accaea/users/6c6b919537074a5eb052e59bf9fc393c/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:28.256141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1573b0c-e034-4e50-965f-7f07de6c10fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=66554ec870b7428ca6a1226686accaea, user_id=6c6b919537074a5eb052e59bf9fc393c, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:28.258832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1573b0c-e034-4e50-965f-7f07de6c10fd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:28.352129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 883/1779] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:28 2022] PUT /identity/v3/projects/66554ec870b7428ca6a1226686accaea/users/6c6b919537074a5eb052e59bf9fc393c/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:28.363092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840ff9-c437-4cd6-a1ac-bd753f62d062 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:28.363703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840ff9-c437-4cd6-a1ac-bd753f62d062 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:28.363944 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840ff9-c437-4cd6-a1ac-bd753f62d062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:28.435170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e4840ff9-c437-4cd6-a1ac-bd753f62d062 None None] MFA Rules not processed for user `6c6b919537074a5eb052e59bf9fc393c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:28.436526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e4840ff9-c437-4cd6-a1ac-bd753f62d062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:28.663346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 897/1780] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:28.669036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1573b0c-e034-4e50-965f-7f07de6c10fd admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:28.884476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1573b0c-e034-4e50-965f-7f07de6c10fd admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:28.891002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315746a-869e-4762-aa64-d0f795828ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:28.892792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315746a-869e-4762-aa64-d0f795828ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:28.892792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315746a-869e-4762-aa64-d0f795828ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:28.892792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315746a-869e-4762-aa64-d0f795828ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:28.893159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315746a-869e-4762-aa64-d0f795828ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:28.895300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315746a-869e-4762-aa64-d0f795828ea7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:29.059280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:29.067403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 884/1781] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:28 2022] POST /identity/v3/projects => generated 385 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:29.071495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4840ff9-c437-4cd6-a1ac-bd753f62d062 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:29.085470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4840ff9-c437-4cd6-a1ac-bd753f62d062 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:29.157862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42ab5f-aa41-4103-94fd-83b9112bb536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:29.159802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42ab5f-aa41-4103-94fd-83b9112bb536 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:29.160399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42ab5f-aa41-4103-94fd-83b9112bb536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:29.160874 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42ab5f-aa41-4103-94fd-83b9112bb536 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:29.161886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42ab5f-aa41-4103-94fd-83b9112bb536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:29.164633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42ab5f-aa41-4103-94fd-83b9112bb536 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:29.293808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 898/1782] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:29 2022] POST /identity/v3/users => generated 299 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:29.298279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315746a-869e-4762-aa64-d0f795828ea7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:29.309263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315746a-869e-4762-aa64-d0f795828ea7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:29.314387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37194a1-14b3-4a55-8f94-4dd5826a8c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:29.315588 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37194a1-14b3-4a55-8f94-4dd5826a8c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:29.315881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37194a1-14b3-4a55-8f94-4dd5826a8c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:29.316163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37194a1-14b3-4a55-8f94-4dd5826a8c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:29.317084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37194a1-14b3-4a55-8f94-4dd5826a8c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:29.318888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37194a1-14b3-4a55-8f94-4dd5826a8c8c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:29.334814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 885/1783] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:29.336900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42ab5f-aa41-4103-94fd-83b9112bb536 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:29.348925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42ab5f-aa41-4103-94fd-83b9112bb536 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:29.354051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19575ad9-c83d-46a8-93c6-31e75da3147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:29.354963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19575ad9-c83d-46a8-93c6-31e75da3147b admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:29.355251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19575ad9-c83d-46a8-93c6-31e75da3147b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:29.355533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19575ad9-c83d-46a8-93c6-31e75da3147b admin admin] PATH_INFO: `/v3/projects/b04848290ae74e9aaf2798d0509d68e2/users/ff3a310200c64b3eb030f47c28195b12/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:29.374746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19575ad9-c83d-46a8-93c6-31e75da3147b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04848290ae74e9aaf2798d0509d68e2, user_id=ff3a310200c64b3eb030f47c28195b12, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:29.379866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19575ad9-c83d-46a8-93c6-31e75da3147b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:29.484949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 899/1784] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:29 2022] PUT /identity/v3/projects/b04848290ae74e9aaf2798d0509d68e2/users/ff3a310200c64b3eb030f47c28195b12/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:29.489929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37194a1-14b3-4a55-8f94-4dd5826a8c8c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:29.503949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37194a1-14b3-4a55-8f94-4dd5826a8c8c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:29.595233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a4f694-e745-417b-870b-f0f495207f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:29.596634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4f694-e745-417b-870b-f0f495207f9e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:29.597407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4f694-e745-417b-870b-f0f495207f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:29.597860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4f694-e745-417b-870b-f0f495207f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:29.599291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4f694-e745-417b-870b-f0f495207f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:29.601729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a4f694-e745-417b-870b-f0f495207f9e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:29.615734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 886/1785] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:29.619960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19575ad9-c83d-46a8-93c6-31e75da3147b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:29.633701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19575ad9-c83d-46a8-93c6-31e75da3147b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:29.641301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:29.642494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:29.643873 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:29.644365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] PATH_INFO: `/v3/projects/b04848290ae74e9aaf2798d0509d68e2/users/ff3a310200c64b3eb030f47c28195b12/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:29.647549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04848290ae74e9aaf2798d0509d68e2, user_id=ff3a310200c64b3eb030f47c28195b12, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:29.650688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:29.672488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 900/1786] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:29 2022] PUT /identity/v3/projects/b04848290ae74e9aaf2798d0509d68e2/users/ff3a310200c64b3eb030f47c28195b12/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:29.684446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf047d-7b7c-415b-909f-ca9bd86c5bb6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:29.685177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf047d-7b7c-415b-909f-ca9bd86c5bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:29.685620 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf047d-7b7c-415b-909f-ca9bd86c5bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:29.735727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e8cf047d-7b7c-415b-909f-ca9bd86c5bb6 None None] MFA Rules not processed for user `ff3a310200c64b3eb030f47c28195b12`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:29.737899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e8cf047d-7b7c-415b-909f-ca9bd86c5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:29.850342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 887/1787] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:29.863384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:29.865394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:29.867095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:29.938780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2d7cf-be18-4992-8c3a-f63e1ec9df5d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:29.941569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:29.942333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:29.943102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:29.943434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:30.012970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:30.014596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:30.025592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 901/1788] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:37.234602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf047d-7b7c-415b-909f-ca9bd86c5bb6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:37.236580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e8cf047d-7b7c-415b-909f-ca9bd86c5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:37.249716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf047d-7b7c-415b-909f-ca9bd86c5bb6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:37.253525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52197e4b-a91f-4c34-8c8a-f801d33ba9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:37.254287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52197e4b-a91f-4c34-8c8a-f801d33ba9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:37.254575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52197e4b-a91f-4c34-8c8a-f801d33ba9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:37.254856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52197e4b-a91f-4c34-8c8a-f801d33ba9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:37.265049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52197e4b-a91f-4c34-8c8a-f801d33ba9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:37.266441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52197e4b-a91f-4c34-8c8a-f801d33ba9d5 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:37.272962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:37.281414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 888/1789] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:37.497302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:37.498859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:37.578468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c7b45-1a79-4ba5-ab6d-968e5d4c16ae service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:37.584381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:37.585494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:37.586110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:37.586531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] PATH_INFO: `/v3/users/c5f14a3e9af24525a940cd2b0a047830` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:37.587923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f14a3e9af24525a940cd2b0a047830)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:37.590312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:37.745347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f14a3e9af24525a940cd2b0a047830'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:37.753435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f14a3e9af24525a940cd2b0a047830'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:37.754635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:37.766304 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f14a3e9af24525a940cd2b0a047830'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:37.767035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f14a3e9af24525a940cd2b0a047830'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:37.844072 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f14a3e9af24525a940cd2b0a047830'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:37.856342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 902/1790] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:37 2022] DELETE /identity/v3/users/c5f14a3e9af24525a940cd2b0a047830 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:37.861067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855de5-e1b6-45f9-90ab-898667a56931 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:37.861998 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855de5-e1b6-45f9-90ab-898667a56931 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:37.862348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73855de5-e1b6-45f9-90ab-898667a56931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:37.885440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-73855de5-e1b6-45f9-90ab-898667a56931 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:37.886824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-73855de5-e1b6-45f9-90ab-898667a56931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:37.964180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 889/1791] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:37.981317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:37.982723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:38.061091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bf9c1-7d69-4371-a254-1292e6de0f82 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:38.065464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fd86-b43e-4df8-96ff-b8ffd4c864a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:38.066535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fd86-b43e-4df8-96ff-b8ffd4c864a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:38.067013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fd86-b43e-4df8-96ff-b8ffd4c864a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:38.067434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fd86-b43e-4df8-96ff-b8ffd4c864a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:38.077561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fd86-b43e-4df8-96ff-b8ffd4c864a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:38.079004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fd86-b43e-4df8-96ff-b8ffd4c864a3 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:38.090761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 903/1792] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:38.896787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73855de5-e1b6-45f9-90ab-898667a56931 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:38.898366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-73855de5-e1b6-45f9-90ab-898667a56931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:38.930797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73855de5-e1b6-45f9-90ab-898667a56931 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:38.939135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:38.941964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:38.942605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:38.943087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] PATH_INFO: `/v3/projects/9e879c61f5ef4544b2f2774817e19694` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:38.944983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e879c61f5ef4544b2f2774817e19694)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:38.950163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:39.095319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e879c61f5ef4544b2f2774817e19694', 'request_id': 'req-f7943241-e5f9-47df-ae63-b417496f0f7e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.102871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e879c61f5ef4544b2f2774817e19694', 'request_id': 'req-f7943241-e5f9-47df-ae63-b417496f0f7e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.110088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e879c61f5ef4544b2f2774817e19694', 'request_id': 'req-f7943241-e5f9-47df-ae63-b417496f0f7e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.209722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] The token cache is being invalidate because project 9e879c61f5ef4544b2f2774817e19694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:39.210019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.210658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.212291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 890/1793] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:38 2022] DELETE /identity/v3/projects/9e879c61f5ef4544b2f2774817e19694 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:39.220227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fd86-b43e-4df8-96ff-b8ffd4c864a3 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:39.227927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fd86-b43e-4df8-96ff-b8ffd4c864a3 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:39.331580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:39.332440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:39.332728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:39.333012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] PATH_INFO: `/v3/users/47bb4ac989ce4d0f8a75fd382d1addf2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:39.334252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47bb4ac989ce4d0f8a75fd382d1addf2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:39.336218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:39.655086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47bb4ac989ce4d0f8a75fd382d1addf2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.665815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47bb4ac989ce4d0f8a75fd382d1addf2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.688722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47bb4ac989ce4d0f8a75fd382d1addf2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.689597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47bb4ac989ce4d0f8a75fd382d1addf2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.694834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:39.807609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4157198e-d33d-4acf-8d89-ad5a925f198e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47bb4ac989ce4d0f8a75fd382d1addf2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:39.833257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 904/1794] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:39 2022] DELETE /identity/v3/users/47bb4ac989ce4d0f8a75fd382d1addf2 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:40.441340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:40.443028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:40.457303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7943241-e5f9-47df-ae63-b417496f0f7e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:40.521909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:40.522780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:40.524601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:40.524807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] PATH_INFO: `/v3/projects/223b1d493a044cdc9d1718eb9f7e2f20` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:40.527233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] RBAC: Authorizing `identity:delete_project(project_id=223b1d493a044cdc9d1718eb9f7e2f20)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:40.527992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:40.554871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:40.654278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223b1d493a044cdc9d1718eb9f7e2f20', 'request_id': 'req-9731ddbe-81e3-4d46-8b52-3eacc325b43f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:40.663450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223b1d493a044cdc9d1718eb9f7e2f20', 'request_id': 'req-9731ddbe-81e3-4d46-8b52-3eacc325b43f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:40.670924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223b1d493a044cdc9d1718eb9f7e2f20', 'request_id': 'req-9731ddbe-81e3-4d46-8b52-3eacc325b43f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:40.797264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] The token cache is being invalidate because project 223b1d493a044cdc9d1718eb9f7e2f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:40.798089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:40.799920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9731ddbe-81e3-4d46-8b52-3eacc325b43f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:40.801655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 891/1795] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:40 2022] DELETE /identity/v3/projects/223b1d493a044cdc9d1718eb9f7e2f20 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:40.808334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7572b-30be-4fa5-b748-84a48c34f6a7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:40.809136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7572b-30be-4fa5-b748-84a48c34f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:40.809547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7572b-30be-4fa5-b748-84a48c34f6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:40.832018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-64a7572b-30be-4fa5-b748-84a48c34f6a7 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:40.833356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-64a7572b-30be-4fa5-b748-84a48c34f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:40.917881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 905/1796] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:40.926243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43732d-d346-4152-8305-71712980a830 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:40.926838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43732d-d346-4152-8305-71712980a830 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:40.927187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43732d-d346-4152-8305-71712980a830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:40.949023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dc43732d-d346-4152-8305-71712980a830 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:40.950945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dc43732d-d346-4152-8305-71712980a830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:40.966982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7572b-30be-4fa5-b748-84a48c34f6a7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:40.977772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 892/1797] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:40.985589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7572b-30be-4fa5-b748-84a48c34f6a7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:40.990910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:40.991579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:40.991881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:40.992158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:40.992883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:40.993249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:40.994922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:41.007382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 906/1798] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:36:41.011869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43732d-d346-4152-8305-71712980a830 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:41.022227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43732d-d346-4152-8305-71712980a830 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:41.027266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7906c-b787-4156-880d-53836f9d3b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:41.027914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7906c-b787-4156-880d-53836f9d3b63 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:41.028197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7906c-b787-4156-880d-53836f9d3b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:41.028474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7906c-b787-4156-880d-53836f9d3b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:41.029208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7906c-b787-4156-880d-53836f9d3b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:41.031317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7906c-b787-4156-880d-53836f9d3b63 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:41.124937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 893/1799] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:41 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:41.129858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:41.139872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad28ddf-7b7a-40b2-ad27-0c2cba9d9c21 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:41.202763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227452f3-9520-4bac-9373-3a7d3c0c5be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:41.205731 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227452f3-9520-4bac-9373-3a7d3c0c5be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:41.206027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227452f3-9520-4bac-9373-3a7d3c0c5be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:41.206316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227452f3-9520-4bac-9373-3a7d3c0c5be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:41.207032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227452f3-9520-4bac-9373-3a7d3c0c5be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:41.209046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227452f3-9520-4bac-9373-3a7d3c0c5be4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:41.336036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 907/1800] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:41 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:41.340211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7906c-b787-4156-880d-53836f9d3b63 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:41.350336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7906c-b787-4156-880d-53836f9d3b63 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:41.354812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9be1-55c1-4e17-bab6-d5eef6ce4493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:41.355459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9be1-55c1-4e17-bab6-d5eef6ce4493 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:41.355743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9be1-55c1-4e17-bab6-d5eef6ce4493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:41.356022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d9be1-55c1-4e17-bab6-d5eef6ce4493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:41.356772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9be1-55c1-4e17-bab6-d5eef6ce4493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:41.358556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d9be1-55c1-4e17-bab6-d5eef6ce4493 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:41.370128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 894/1801] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:41.375120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227452f3-9520-4bac-9373-3a7d3c0c5be4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:41.385759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227452f3-9520-4bac-9373-3a7d3c0c5be4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:41.390520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0feb8-85db-4fce-add3-cf80af9ffd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:41.391222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0feb8-85db-4fce-add3-cf80af9ffd4f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:41.391656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0feb8-85db-4fce-add3-cf80af9ffd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:41.391859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0feb8-85db-4fce-add3-cf80af9ffd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:41.392598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0feb8-85db-4fce-add3-cf80af9ffd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:41.394228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0feb8-85db-4fce-add3-cf80af9ffd4f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:41.404710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 908/1802] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:41.408930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9be1-55c1-4e17-bab6-d5eef6ce4493 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:41.418820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d9be1-55c1-4e17-bab6-d5eef6ce4493 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:41.423712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12314f4-2937-42ad-b725-2cab1834445c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:41.425116 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12314f4-2937-42ad-b725-2cab1834445c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:41.425530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12314f4-2937-42ad-b725-2cab1834445c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:41.431788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12314f4-2937-42ad-b725-2cab1834445c admin admin] PATH_INFO: `/v3/projects/26accbd88d5243c4af00a5fb5225aabb/users/23bb05bb2f2442aa84c66c8692b02de8/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:41.441176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12314f4-2937-42ad-b725-2cab1834445c admin admin] RBAC: Authorizing `identity:create_grant(project_id=26accbd88d5243c4af00a5fb5225aabb, user_id=23bb05bb2f2442aa84c66c8692b02de8, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:41.443827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12314f4-2937-42ad-b725-2cab1834445c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:41.534145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 895/1803] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:41 2022] PUT /identity/v3/projects/26accbd88d5243c4af00a5fb5225aabb/users/23bb05bb2f2442aa84c66c8692b02de8/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:41.542703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22022c-18cc-4666-ac75-c3ae8d680a70 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:41.542997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22022c-18cc-4666-ac75-c3ae8d680a70 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:41.543279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22022c-18cc-4666-ac75-c3ae8d680a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:41.571555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0f22022c-18cc-4666-ac75-c3ae8d680a70 None None] MFA Rules not processed for user `23bb05bb2f2442aa84c66c8692b02de8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:41.572819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0f22022c-18cc-4666-ac75-c3ae8d680a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:41.628398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:41.646411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 909/1804] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:41.650932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12314f4-2937-42ad-b725-2cab1834445c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:41.740283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12314f4-2937-42ad-b725-2cab1834445c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:41.745712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b9b33-430e-4808-bfdf-79990c3bec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:41.746395 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b9b33-430e-4808-bfdf-79990c3bec77 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:41.746766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b9b33-430e-4808-bfdf-79990c3bec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:41.747093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b9b33-430e-4808-bfdf-79990c3bec77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:41.753624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b9b33-430e-4808-bfdf-79990c3bec77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:41.756298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b9b33-430e-4808-bfdf-79990c3bec77 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:41.881298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 896/1805] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:41 2022] POST /identity/v3/projects => generated 391 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:41.885724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22022c-18cc-4666-ac75-c3ae8d680a70 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:41.916054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f22022c-18cc-4666-ac75-c3ae8d680a70 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:41.979784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132413c9-4576-4684-856e-f43c24499bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:41.980463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132413c9-4576-4684-856e-f43c24499bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:41.980750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132413c9-4576-4684-856e-f43c24499bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:41.981029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132413c9-4576-4684-856e-f43c24499bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:41.981814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132413c9-4576-4684-856e-f43c24499bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:41.983761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132413c9-4576-4684-856e-f43c24499bb7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:42.079354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 910/1806] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:41 2022] POST /identity/v3/users => generated 302 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:42.083207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b9b33-430e-4808-bfdf-79990c3bec77 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:42.098797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b9b33-430e-4808-bfdf-79990c3bec77 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:42.104273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bbd88-ac5b-4eaa-9b57-0d5b9dda3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:42.104948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbd88-ac5b-4eaa-9b57-0d5b9dda3eac admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.105272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbd88-ac5b-4eaa-9b57-0d5b9dda3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.105555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbd88-ac5b-4eaa-9b57-0d5b9dda3eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.106288 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bbd88-ac5b-4eaa-9b57-0d5b9dda3eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:42.107944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bbd88-ac5b-4eaa-9b57-0d5b9dda3eac admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:42.120510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 897/1807] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:42.124837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132413c9-4576-4684-856e-f43c24499bb7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:42.136089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132413c9-4576-4684-856e-f43c24499bb7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:42.140911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:42.141871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.142182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.142467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] PATH_INFO: `/v3/projects/5d2d078062914a7fb9bb653be52ca36d/users/ec5079ce2bd54ea59523ff08bc1d8cc6/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.159266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2d078062914a7fb9bb653be52ca36d, user_id=ec5079ce2bd54ea59523ff08bc1d8cc6, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:42.161984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:42.246414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 911/1808] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:42 2022] PUT /identity/v3/projects/5d2d078062914a7fb9bb653be52ca36d/users/ec5079ce2bd54ea59523ff08bc1d8cc6/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:42.250575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bbd88-ac5b-4eaa-9b57-0d5b9dda3eac admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:42.262224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bbd88-ac5b-4eaa-9b57-0d5b9dda3eac admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:42.281185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:42.328308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89d0ff-7508-418e-9f6c-46da6210d583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:42.331949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89d0ff-7508-418e-9f6c-46da6210d583 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.331949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89d0ff-7508-418e-9f6c-46da6210d583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.332326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c89d0ff-7508-418e-9f6c-46da6210d583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.333132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89d0ff-7508-418e-9f6c-46da6210d583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:42.334765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c89d0ff-7508-418e-9f6c-46da6210d583 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:42.342384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:42.344879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:42.353209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 898/1809] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:42.357411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89d0ff-7508-418e-9f6c-46da6210d583 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:42.364728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a29db-80d8-47ed-95a8-a588632be4c9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:42.367959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c89d0ff-7508-418e-9f6c-46da6210d583 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:42.370693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:42.371881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.372371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.372900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] PATH_INFO: `/v3/users/7a6af81a87d8464999fa02d8a5d20f6a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.373277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295be1a-9fa2-467b-9e2b-1f7a26664c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:42.373994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295be1a-9fa2-467b-9e2b-1f7a26664c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.374677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295be1a-9fa2-467b-9e2b-1f7a26664c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.374963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295be1a-9fa2-467b-9e2b-1f7a26664c69 admin admin] PATH_INFO: `/v3/projects/5d2d078062914a7fb9bb653be52ca36d/users/ec5079ce2bd54ea59523ff08bc1d8cc6/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.376110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6af81a87d8464999fa02d8a5d20f6a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:42.376988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295be1a-9fa2-467b-9e2b-1f7a26664c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2d078062914a7fb9bb653be52ca36d, user_id=ec5079ce2bd54ea59523ff08bc1d8cc6, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:42.379031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:42.380328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295be1a-9fa2-467b-9e2b-1f7a26664c69 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:42.394804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 899/1810] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:42 2022] PUT /identity/v3/projects/5d2d078062914a7fb9bb653be52ca36d/users/ec5079ce2bd54ea59523ff08bc1d8cc6/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:42.408127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187397-0a50-4f43-b871-0e3e40095291 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.408491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187397-0a50-4f43-b871-0e3e40095291 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.408781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187397-0a50-4f43-b871-0e3e40095291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.434531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f2187397-0a50-4f43-b871-0e3e40095291 None None] MFA Rules not processed for user `ec5079ce2bd54ea59523ff08bc1d8cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:42.435736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f2187397-0a50-4f43-b871-0e3e40095291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:42.504296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 900/1811] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:42.518820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2187397-0a50-4f43-b871-0e3e40095291 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:42.520142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f2187397-0a50-4f43-b871-0e3e40095291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:42.574461 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2187397-0a50-4f43-b871-0e3e40095291 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:42.578213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:42.578800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.579089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.579519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.589465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:42.590905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:42.600792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 901/1812] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:42.653367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6af81a87d8464999fa02d8a5d20f6a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:42.654814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:42.656237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:42.660649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6af81a87d8464999fa02d8a5d20f6a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:42.667171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24c616-a716-4b1b-a67d-abd6dd7f7841 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:42.670964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e16a32-a5ee-465a-a3e0-bf7c3c1f447d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:42.671595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e16a32-a5ee-465a-a3e0-bf7c3c1f447d service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.671877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e16a32-a5ee-465a-a3e0-bf7c3c1f447d service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.672152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e16a32-a5ee-465a-a3e0-bf7c3c1f447d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.673668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6af81a87d8464999fa02d8a5d20f6a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:42.674907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6af81a87d8464999fa02d8a5d20f6a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:42.719345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:42.734028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e16a32-a5ee-465a-a3e0-bf7c3c1f447d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:42.735415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e16a32-a5ee-465a-a3e0-bf7c3c1f447d service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:42.748795 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 902/1813] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:42.770737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6af81a87d8464999fa02d8a5d20f6a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:42.783260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 912/1814] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:42 2022] DELETE /identity/v3/users/7a6af81a87d8464999fa02d8a5d20f6a => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:42.784852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4f670-f122-4dcd-b128-464aac709a9d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.785188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4f670-f122-4dcd-b128-464aac709a9d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.785532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4f670-f122-4dcd-b128-464aac709a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.819254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-22c4f670-f122-4dcd-b128-464aac709a9d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:42.821096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-22c4f670-f122-4dcd-b128-464aac709a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:42.887622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 903/1815] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:42.897234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:42.898693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:42.942366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:42.967308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0e02a-ea3a-47cb-a38b-750fef0cc910 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:42.971799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:42.972457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:42.972726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:42.973000 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:42.982443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:42.983901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:43.007217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 913/1816] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:43.477511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4f670-f122-4dcd-b128-464aac709a9d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:43.479157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-22c4f670-f122-4dcd-b128-464aac709a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:43.492811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4f670-f122-4dcd-b128-464aac709a9d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:43.498499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:43.499390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:43.499691 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:43.504177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] PATH_INFO: `/v3/projects/8fd94d5e12cc4aefa06f07f7884ce865` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:43.504177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd94d5e12cc4aefa06f07f7884ce865)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:43.504177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:43.630004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd94d5e12cc4aefa06f07f7884ce865', 'request_id': 'req-0e0eb156-2050-4792-9850-53acae75cde6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:43.638381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd94d5e12cc4aefa06f07f7884ce865', 'request_id': 'req-0e0eb156-2050-4792-9850-53acae75cde6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:43.644316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd94d5e12cc4aefa06f07f7884ce865', 'request_id': 'req-0e0eb156-2050-4792-9850-53acae75cde6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:43.659025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:43.659025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:43.678272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bb895-dd46-42bd-9a30-b9f3859e85bf service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:43.741714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:43.742603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:43.742891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:43.743171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] PATH_INFO: `/v3/users/6c6b919537074a5eb052e59bf9fc393c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:43.744892 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c6b919537074a5eb052e59bf9fc393c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:43.747101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:43.753603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] The token cache is being invalidate because project 8fd94d5e12cc4aefa06f07f7884ce865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:43.753890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:43.754566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:43.756385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 904/1817] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:43 2022] DELETE /identity/v3/projects/8fd94d5e12cc4aefa06f07f7884ce865 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:43.759970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:43.761669 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:43.778438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eb156-2050-4792-9850-53acae75cde6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:43.784280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:43.785219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:43.785505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:43.785785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] PATH_INFO: `/v3/users/c2ef777b15fc4b16ae6ddb1f4784dc1b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:43.787121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ef777b15fc4b16ae6ddb1f4784dc1b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:43.789203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:43.941054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c6b919537074a5eb052e59bf9fc393c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:43.948534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c6b919537074a5eb052e59bf9fc393c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:43.961667 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c6b919537074a5eb052e59bf9fc393c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:43.962406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c6b919537074a5eb052e59bf9fc393c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.022837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ef777b15fc4b16ae6ddb1f4784dc1b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.024735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:44.033203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b29393ee-9342-4846-b1b5-8e862fdb2536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c6b919537074a5eb052e59bf9fc393c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.037214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ef777b15fc4b16ae6ddb1f4784dc1b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.038217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ef777b15fc4b16ae6ddb1f4784dc1b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.038833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ef777b15fc4b16ae6ddb1f4784dc1b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.040522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 914/1818] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:43 2022] DELETE /identity/v3/users/6c6b919537074a5eb052e59bf9fc393c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:44.045009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee56a-0e65-4389-8c8a-2d47bce99d9d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:44.045334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee56a-0e65-4389-8c8a-2d47bce99d9d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:44.045619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcee56a-0e65-4389-8c8a-2d47bce99d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:44.064393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1fcee56a-0e65-4389-8c8a-2d47bce99d9d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:44.065695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1fcee56a-0e65-4389-8c8a-2d47bce99d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:44.116588 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ef777b15fc4b16ae6ddb1f4784dc1b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.117524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:44.144353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 905/1819] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:43 2022] DELETE /identity/v3/users/c2ef777b15fc4b16ae6ddb1f4784dc1b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:44.151520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 915/1820] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:44.161728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:44.163119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:44.220437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337320b-5a43-40b0-a30d-10a9e48ffa5f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:44.224706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:44.225389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:44.225675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:44.225951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:44.289674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:44.291113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:44.302349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 906/1821] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:44.687554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcee56a-0e65-4389-8c8a-2d47bce99d9d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:44.691307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1fcee56a-0e65-4389-8c8a-2d47bce99d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:44.706566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcee56a-0e65-4389-8c8a-2d47bce99d9d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:44.711862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:44.719015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:44.719384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:44.719743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] PATH_INFO: `/v3/projects/c2f33339c845430a8bc86b5a56cba1c9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:44.722329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f33339c845430a8bc86b5a56cba1c9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:44.724517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:44.919854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:44.925120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f33339c845430a8bc86b5a56cba1c9', 'request_id': 'req-0f549113-94b5-431b-9747-128ecf18a772'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.925813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:44.935591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f33339c845430a8bc86b5a56cba1c9', 'request_id': 'req-0f549113-94b5-431b-9747-128ecf18a772'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.943165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f33339c845430a8bc86b5a56cba1c9', 'request_id': 'req-0f549113-94b5-431b-9747-128ecf18a772'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:44.954941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6a83-8b35-4bfb-820d-962fe6c50c7e service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:45.019183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:45.020293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.020785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.021157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] PATH_INFO: `/v3/projects/66554ec870b7428ca6a1226686accaea` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.022459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66554ec870b7428ca6a1226686accaea)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:45.024381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:45.027594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] The token cache is being invalidate because project c2f33339c845430a8bc86b5a56cba1c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:45.028199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.029275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0f549113-94b5-431b-9747-128ecf18a772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.032122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 916/1822] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:44 2022] DELETE /identity/v3/projects/c2f33339c845430a8bc86b5a56cba1c9 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:45.037681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60284e-0623-4780-831d-57a2236c911b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.046237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60284e-0623-4780-831d-57a2236c911b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.046237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60284e-0623-4780-831d-57a2236c911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.050262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:45.067015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2d60284e-0623-4780-831d-57a2236c911b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:45.068534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2d60284e-0623-4780-831d-57a2236c911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:45.077093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 917/1823] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:45.084948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477e41f-59e5-428a-b541-0ba21af68132 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.085271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477e41f-59e5-428a-b541-0ba21af68132 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.085552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477e41f-59e5-428a-b541-0ba21af68132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.105724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e477e41f-59e5-428a-b541-0ba21af68132 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:45.106690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e477e41f-59e5-428a-b541-0ba21af68132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:45.116885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 918/1824] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:45.122003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477e41f-59e5-428a-b541-0ba21af68132 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:45.133662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66554ec870b7428ca6a1226686accaea', 'request_id': 'req-2d3dd55f-e8a0-4567-8668-305937ca33c7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.134204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477e41f-59e5-428a-b541-0ba21af68132 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:45.140101 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66554ec870b7428ca6a1226686accaea', 'request_id': 'req-2d3dd55f-e8a0-4567-8668-305937ca33c7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.146486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66554ec870b7428ca6a1226686accaea', 'request_id': 'req-2d3dd55f-e8a0-4567-8668-305937ca33c7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.199799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:45.200385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.200673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.200950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.201870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:45.202293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:45.203894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:45.217223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 919/1825] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:36:45.222208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:45.231739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3f23b-ea38-45b1-84f2-b7c0bad912e3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:45.237615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:45.238274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.238556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.238832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.239313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] The token cache is being invalidate because project 66554ec870b7428ca6a1226686accaea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:45.241217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:45.241316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.241316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.241444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:45.242241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 907/1826] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:44 2022] DELETE /identity/v3/projects/66554ec870b7428ca6a1226686accaea => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:45.247195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:45.248899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:45.266080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dd55f-e8a0-4567-8668-305937ca33c7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:45.270871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:45.271823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.272104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.272377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] PATH_INFO: `/v3/users/ff3a310200c64b3eb030f47c28195b12` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.273670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3a310200c64b3eb030f47c28195b12)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:45.275624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:45.328252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 920/1827] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:45 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:45.335901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:45.337029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:45.384880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:45.419245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ca428-de65-4c89-8225-564903c5a1cc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:45.425280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65625a7-3d13-4864-93db-41f01335b1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:45.426185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65625a7-3d13-4864-93db-41f01335b1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.426493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65625a7-3d13-4864-93db-41f01335b1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.426786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65625a7-3d13-4864-93db-41f01335b1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.427495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65625a7-3d13-4864-93db-41f01335b1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:45.429546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65625a7-3d13-4864-93db-41f01335b1b5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:45.454515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3a310200c64b3eb030f47c28195b12'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.472656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3a310200c64b3eb030f47c28195b12'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.473680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3a310200c64b3eb030f47c28195b12'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.473680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3a310200c64b3eb030f47c28195b12'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.541925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3a310200c64b3eb030f47c28195b12'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:45.559660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 908/1828] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:45 2022] DELETE /identity/v3/users/ff3a310200c64b3eb030f47c28195b12 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:45.668859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 921/1829] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:45 2022] POST /identity/v3/users => generated 283 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:45.673164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:45.675138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:45.752829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244728-081a-4db5-b901-9dee697934f7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:45.758195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78df9b0-3b91-4200-b518-55845cec0ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:45.758870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78df9b0-3b91-4200-b518-55845cec0ecf admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.759215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78df9b0-3b91-4200-b518-55845cec0ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.769290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78df9b0-3b91-4200-b518-55845cec0ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.770192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78df9b0-3b91-4200-b518-55845cec0ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:45.771805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78df9b0-3b91-4200-b518-55845cec0ecf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:45.787357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 909/1830] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:45.791683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65625a7-3d13-4864-93db-41f01335b1b5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:45.813366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65625a7-3d13-4864-93db-41f01335b1b5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:45.828523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97a3f5-f7f5-40ec-9f9d-8a518a8d6ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:45.842902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97a3f5-f7f5-40ec-9f9d-8a518a8d6ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.843202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97a3f5-f7f5-40ec-9f9d-8a518a8d6ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.845601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97a3f5-f7f5-40ec-9f9d-8a518a8d6ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.849599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97a3f5-f7f5-40ec-9f9d-8a518a8d6ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:45.852636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97a3f5-f7f5-40ec-9f9d-8a518a8d6ad1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:45.866924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 922/1831] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:45.871010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78df9b0-3b91-4200-b518-55845cec0ecf admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:45.883025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78df9b0-3b91-4200-b518-55845cec0ecf admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:45.887875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:45.888812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:45.889618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:45.889907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] PATH_INFO: `/v3/projects/ec50599e2b7b4e958c0cf261b17eed02/users/90d4f9a74b0f4a61a7f0e6b57bcff62e/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:45.907034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec50599e2b7b4e958c0cf261b17eed02, user_id=90d4f9a74b0f4a61a7f0e6b57bcff62e, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:45.912207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:45.932352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:46.021496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 910/1832] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:45 2022] PUT /identity/v3/projects/ec50599e2b7b4e958c0cf261b17eed02/users/90d4f9a74b0f4a61a7f0e6b57bcff62e/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:46.032438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342463-96de-4ecd-82d3-a19efc1714d0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.033013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342463-96de-4ecd-82d3-a19efc1714d0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.033013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1342463-96de-4ecd-82d3-a19efc1714d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.061149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.062963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:46.069551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b1342463-96de-4ecd-82d3-a19efc1714d0 None None] MFA Rules not processed for user `90d4f9a74b0f4a61a7f0e6b57bcff62e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:46.071923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b1342463-96de-4ecd-82d3-a19efc1714d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:46.138368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4461356-b7c2-4771-a991-c7f25d67b41d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.142939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 923/1833] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:46.143907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.144850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.145234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.145575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] PATH_INFO: `/v3/projects/b04848290ae74e9aaf2798d0509d68e2` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.147274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04848290ae74e9aaf2798d0509d68e2)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.147401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342463-96de-4ecd-82d3-a19efc1714d0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.149085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.158962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1342463-96de-4ecd-82d3-a19efc1714d0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.163632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348d6a4-9d7e-49d5-8c6c-9446b15e10ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.164293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348d6a4-9d7e-49d5-8c6c-9446b15e10ed admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.164580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348d6a4-9d7e-49d5-8c6c-9446b15e10ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.164862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348d6a4-9d7e-49d5-8c6c-9446b15e10ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.165697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348d6a4-9d7e-49d5-8c6c-9446b15e10ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.167642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348d6a4-9d7e-49d5-8c6c-9446b15e10ed admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.282600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 924/1834] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:46 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:46.286695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348d6a4-9d7e-49d5-8c6c-9446b15e10ed admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.297281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04848290ae74e9aaf2798d0509d68e2', 'request_id': 'req-2815bc1a-c655-4fa4-b07a-338c7bc20cea'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:46.305721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04848290ae74e9aaf2798d0509d68e2', 'request_id': 'req-2815bc1a-c655-4fa4-b07a-338c7bc20cea'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:46.311303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04848290ae74e9aaf2798d0509d68e2', 'request_id': 'req-2815bc1a-c655-4fa4-b07a-338c7bc20cea'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:46.358439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] The token cache is being invalidate because project b04848290ae74e9aaf2798d0509d68e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:46.358749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:46.359448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2815bc1a-c655-4fa4-b07a-338c7bc20cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:46.361267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 911/1835] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:46 2022] DELETE /identity/v3/projects/b04848290ae74e9aaf2798d0509d68e2 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:46.366450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348d6a4-9d7e-49d5-8c6c-9446b15e10ed admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.367733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae4c3-a559-482e-a391-f36ee11acecc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.368032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae4c3-a559-482e-a391-f36ee11acecc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.369328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ae4c3-a559-482e-a391-f36ee11acecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.372241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.372920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.373240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.373525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.374309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.376440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.390867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b08ae4c3-a559-482e-a391-f36ee11acecc None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:46.396439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b08ae4c3-a559-482e-a391-f36ee11acecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:46.405852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 912/1836] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:46.414347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc491b-0c13-42ca-a092-1f1c99dd347d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.415445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc491b-0c13-42ca-a092-1f1c99dd347d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.415445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc491b-0c13-42ca-a092-1f1c99dd347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.434453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b8bc491b-0c13-42ca-a092-1f1c99dd347d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:46.435805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b8bc491b-0c13-42ca-a092-1f1c99dd347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:46.444810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 913/1837] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:46.449033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc491b-0c13-42ca-a092-1f1c99dd347d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.459063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc491b-0c13-42ca-a092-1f1c99dd347d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.465893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 925/1838] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:46 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:46.466057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.466358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.466642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.467035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.467362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.467776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:46.468092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.469423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:46.470329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.482878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083a036-8a2b-4664-9f03-37ce3dd81a2c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.488793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc487de-c638-4adc-8c70-350025a329f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.489470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc487de-c638-4adc-8c70-350025a329f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.489764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc487de-c638-4adc-8c70-350025a329f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.490075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc487de-c638-4adc-8c70-350025a329f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.490823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc487de-c638-4adc-8c70-350025a329f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.492432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc487de-c638-4adc-8c70-350025a329f6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.493302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 914/1839] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:36:46.497736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.498919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:46.508343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf96d0-9b5e-4d97-85ae-47bc99ae755c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.510475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 926/1840] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:46.513652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac4f29-08f6-437f-92c6-aca32d70f912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.514287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac4f29-08f6-437f-92c6-aca32d70f912 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.514573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac4f29-08f6-437f-92c6-aca32d70f912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.514670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc487de-c638-4adc-8c70-350025a329f6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.514851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac4f29-08f6-437f-92c6-aca32d70f912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.515536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac4f29-08f6-437f-92c6-aca32d70f912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.517508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac4f29-08f6-437f-92c6-aca32d70f912 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.526554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc487de-c638-4adc-8c70-350025a329f6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.532582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57033f0-e654-493e-aab1-94ed3cccb728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.534081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57033f0-e654-493e-aab1-94ed3cccb728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.534589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57033f0-e654-493e-aab1-94ed3cccb728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.534867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57033f0-e654-493e-aab1-94ed3cccb728 admin admin] PATH_INFO: `/v3/projects/93ba0eb6469d4d779436605f40642983/users/944c9bdc04bb44429d62857fa85601da/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.550443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57033f0-e654-493e-aab1-94ed3cccb728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ba0eb6469d4d779436605f40642983, user_id=944c9bdc04bb44429d62857fa85601da, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.553163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57033f0-e654-493e-aab1-94ed3cccb728 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.601817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 915/1841] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:46 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:46.605795 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac4f29-08f6-437f-92c6-aca32d70f912 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.657647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 927/1842] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:46 2022] PUT /identity/v3/projects/93ba0eb6469d4d779436605f40642983/users/944c9bdc04bb44429d62857fa85601da/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:46.660079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57033f0-e654-493e-aab1-94ed3cccb728 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.681422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac4f29-08f6-437f-92c6-aca32d70f912 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.771687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce43edd-a854-44b6-b7a5-e52b627b028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.772519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce43edd-a854-44b6-b7a5-e52b627b028b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.772806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce43edd-a854-44b6-b7a5-e52b627b028b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.773079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce43edd-a854-44b6-b7a5-e52b627b028b admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.773807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce43edd-a854-44b6-b7a5-e52b627b028b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.777690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57033f0-e654-493e-aab1-94ed3cccb728 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.778368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce43edd-a854-44b6-b7a5-e52b627b028b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.784074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642e5e5-f825-4f45-b72b-b1ad9c0a3130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.784449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642e5e5-f825-4f45-b72b-b1ad9c0a3130 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.784809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642e5e5-f825-4f45-b72b-b1ad9c0a3130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.785190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642e5e5-f825-4f45-b72b-b1ad9c0a3130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.787218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642e5e5-f825-4f45-b72b-b1ad9c0a3130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.791482 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642e5e5-f825-4f45-b72b-b1ad9c0a3130 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.800887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:46.803473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 928/1843] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:46.807808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642e5e5-f825-4f45-b72b-b1ad9c0a3130 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.820003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642e5e5-f825-4f45-b72b-b1ad9c0a3130 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.825174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed8bdb-0b3b-40e5-9f6e-c1151c72869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:46.826452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed8bdb-0b3b-40e5-9f6e-c1151c72869e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.826684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed8bdb-0b3b-40e5-9f6e-c1151c72869e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.827222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed8bdb-0b3b-40e5-9f6e-c1151c72869e admin admin] PATH_INFO: `/v3/projects/93ba0eb6469d4d779436605f40642983/users/944c9bdc04bb44429d62857fa85601da/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.829588 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed8bdb-0b3b-40e5-9f6e-c1151c72869e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ba0eb6469d4d779436605f40642983, user_id=944c9bdc04bb44429d62857fa85601da, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:46.833144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed8bdb-0b3b-40e5-9f6e-c1151c72869e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:46.847006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 929/1844] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:46 2022] PUT /identity/v3/projects/93ba0eb6469d4d779436605f40642983/users/944c9bdc04bb44429d62857fa85601da/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:46.856950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a997-3a84-4c60-b366-1c4156284980 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:46.857289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a997-3a84-4c60-b366-1c4156284980 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:46.857570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3a997-3a84-4c60-b366-1c4156284980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:46.895491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f0d3a997-3a84-4c60-b366-1c4156284980 None None] MFA Rules not processed for user `944c9bdc04bb44429d62857fa85601da`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:46.896931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f0d3a997-3a84-4c60-b366-1c4156284980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:46.922225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 916/1845] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:46 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:46.926538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce43edd-a854-44b6-b7a5-e52b627b028b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:46.940464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce43edd-a854-44b6-b7a5-e52b627b028b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:46.993653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 930/1846] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:47.007430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3a997-3a84-4c60-b366-1c4156284980 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.009794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f0d3a997-3a84-4c60-b366-1c4156284980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:47.015430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317869e-ad43-4498-8fec-75b54468b862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.016320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317869e-ad43-4498-8fec-75b54468b862 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.016762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317869e-ad43-4498-8fec-75b54468b862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.018335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317869e-ad43-4498-8fec-75b54468b862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.018796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317869e-ad43-4498-8fec-75b54468b862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.021599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317869e-ad43-4498-8fec-75b54468b862 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.043110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 917/1847] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:47.044360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317869e-ad43-4498-8fec-75b54468b862 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.056620 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317869e-ad43-4498-8fec-75b54468b862 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.062628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfd7d7-7f9a-4b67-8f04-99bf78e436ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.063580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfd7d7-7f9a-4b67-8f04-99bf78e436ad admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.063976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfd7d7-7f9a-4b67-8f04-99bf78e436ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.064359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfd7d7-7f9a-4b67-8f04-99bf78e436ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.065786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfd7d7-7f9a-4b67-8f04-99bf78e436ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.067699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfd7d7-7f9a-4b67-8f04-99bf78e436ad admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.086441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 918/1848] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:47.087630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfd7d7-7f9a-4b67-8f04-99bf78e436ad admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.102521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfd7d7-7f9a-4b67-8f04-99bf78e436ad admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.115836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336764e-ad35-4e81-a1aa-c5baa4030321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.118138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336764e-ad35-4e81-a1aa-c5baa4030321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.118543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336764e-ad35-4e81-a1aa-c5baa4030321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.118917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336764e-ad35-4e81-a1aa-c5baa4030321 admin admin] PATH_INFO: `/v3/projects/e2b15be296924b19b3d68cfc802b4174/users/36e7565ddb574fbc874d46fb80b881a7/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.140058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:47.151661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336764e-ad35-4e81-a1aa-c5baa4030321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b15be296924b19b3d68cfc802b4174, user_id=36e7565ddb574fbc874d46fb80b881a7, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.160874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336764e-ad35-4e81-a1aa-c5baa4030321 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.164059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3a997-3a84-4c60-b366-1c4156284980 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.168826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbecca-c707-4403-b41b-a2fc1ad88d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.169633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbecca-c707-4403-b41b-a2fc1ad88d7d service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.169984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbecca-c707-4403-b41b-a2fc1ad88d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.170368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbecca-c707-4403-b41b-a2fc1ad88d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.182336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbecca-c707-4403-b41b-a2fc1ad88d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.183787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbecca-c707-4403-b41b-a2fc1ad88d7d service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.201337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 931/1849] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:47 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:47.277174 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 919/1850] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:47 2022] PUT /identity/v3/projects/e2b15be296924b19b3d68cfc802b4174/users/36e7565ddb574fbc874d46fb80b881a7/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:47.287928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a473b-d46a-4d60-b8cf-5c1091cb90f8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.288325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a473b-d46a-4d60-b8cf-5c1091cb90f8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.288672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a473b-d46a-4d60-b8cf-5c1091cb90f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.330922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4c5a473b-d46a-4d60-b8cf-5c1091cb90f8 None None] MFA Rules not processed for user `36e7565ddb574fbc874d46fb80b881a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:47.333154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4c5a473b-d46a-4d60-b8cf-5c1091cb90f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:47.437045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 932/1851] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:47.445060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336764e-ad35-4e81-a1aa-c5baa4030321 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.446410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a473b-d46a-4d60-b8cf-5c1091cb90f8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.488178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:47.529791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336764e-ad35-4e81-a1aa-c5baa4030321 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.533932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.535542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.535886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.536519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.539031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:47.539150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a473b-d46a-4d60-b8cf-5c1091cb90f8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.543940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2e5f6-05f6-416c-aeba-c51e4ebb1dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.544625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e5f6-05f6-416c-aeba-c51e4ebb1dde admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.544916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e5f6-05f6-416c-aeba-c51e4ebb1dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.545230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2e5f6-05f6-416c-aeba-c51e4ebb1dde admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.545938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2e5f6-05f6-416c-aeba-c51e4ebb1dde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.547881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2e5f6-05f6-416c-aeba-c51e4ebb1dde admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.620595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.623080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.634523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 933/1852] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:47 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:47.635189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 920/1853] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:47.655434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2e5f6-05f6-416c-aeba-c51e4ebb1dde admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.769320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2e5f6-05f6-416c-aeba-c51e4ebb1dde admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.774692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebd783-1d4e-4840-8709-a1d21b982661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.775365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebd783-1d4e-4840-8709-a1d21b982661 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.775803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebd783-1d4e-4840-8709-a1d21b982661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.776131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebd783-1d4e-4840-8709-a1d21b982661 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.776884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebd783-1d4e-4840-8709-a1d21b982661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.779098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebd783-1d4e-4840-8709-a1d21b982661 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.786139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:47.839968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 934/1854] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:47 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:47.841211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.853621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca06ad-5b96-4bed-a8ad-cf27765a5333 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.864826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772cbd7-a141-4510-b87d-9fece0fdd55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.865965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772cbd7-a141-4510-b87d-9fece0fdd55c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.866333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772cbd7-a141-4510-b87d-9fece0fdd55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.866686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772cbd7-a141-4510-b87d-9fece0fdd55c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.867545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772cbd7-a141-4510-b87d-9fece0fdd55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.870914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772cbd7-a141-4510-b87d-9fece0fdd55c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.883601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 921/1855] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:47.887622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebd783-1d4e-4840-8709-a1d21b982661 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.903622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebd783-1d4e-4840-8709-a1d21b982661 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.918549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.919474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.919762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.920097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.920786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.922505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.934920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 935/1856] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:47.939198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772cbd7-a141-4510-b87d-9fece0fdd55c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.949810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:47.950020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772cbd7-a141-4510-b87d-9fece0fdd55c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.951987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:47.957487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376c716-cca1-4633-a5a3-fa95de04714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.958384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376c716-cca1-4633-a5a3-fa95de04714a admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.958666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376c716-cca1-4633-a5a3-fa95de04714a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.958941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376c716-cca1-4633-a5a3-fa95de04714a admin admin] PATH_INFO: `/v3/projects/58b1765f61e54a85ae5b3d15d133c9ed/users/83b9d86f338c42fab588f1af709336fc/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.973188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bc88b-636f-4f0b-8480-dc9febdd1335 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:47.978212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:47.979130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:47.979419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:47.979695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] PATH_INFO: `/v3/users/23bb05bb2f2442aa84c66c8692b02de8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:47.979828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376c716-cca1-4633-a5a3-fa95de04714a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b1765f61e54a85ae5b3d15d133c9ed, user_id=83b9d86f338c42fab588f1af709336fc, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.981165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23bb05bb2f2442aa84c66c8692b02de8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:47.982774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376c716-cca1-4633-a5a3-fa95de04714a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:47.983163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:48.053590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 922/1857] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:47 2022] PUT /identity/v3/projects/58b1765f61e54a85ae5b3d15d133c9ed/users/83b9d86f338c42fab588f1af709336fc/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:48.065359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d646b-791e-4332-a6bc-971c863baf65 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.065653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d646b-791e-4332-a6bc-971c863baf65 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.065963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d646b-791e-4332-a6bc-971c863baf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.099544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d87d646b-791e-4332-a6bc-971c863baf65 None None] MFA Rules not processed for user `83b9d86f338c42fab588f1af709336fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:48.100785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d87d646b-791e-4332-a6bc-971c863baf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:48.155787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23bb05bb2f2442aa84c66c8692b02de8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:48.166644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23bb05bb2f2442aa84c66c8692b02de8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:48.184344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 923/1858] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:48.186575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23bb05bb2f2442aa84c66c8692b02de8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:48.187440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23bb05bb2f2442aa84c66c8692b02de8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:48.188938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d646b-791e-4332-a6bc-971c863baf65 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:48.246378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:48.266506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d646b-791e-4332-a6bc-971c863baf65 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:48.271879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:48.272523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.272811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.273091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.273594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc708b05-c9ab-4035-95ae-828d3bf7428a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23bb05bb2f2442aa84c66c8692b02de8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:48.273812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:48.279263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:48.287188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 936/1859] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:47 2022] DELETE /identity/v3/users/23bb05bb2f2442aa84c66c8692b02de8 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:48.297793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690a68-edcb-4673-afc8-85379a3fa46c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.298086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690a68-edcb-4673-afc8-85379a3fa46c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.298570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690a68-edcb-4673-afc8-85379a3fa46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.317023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-35690a68-edcb-4673-afc8-85379a3fa46c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:48.317825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-35690a68-edcb-4673-afc8-85379a3fa46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:48.354860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 924/1860] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:48 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:48.358781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:48.360585 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:48.386579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 937/1861] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:48.397811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35690a68-edcb-4673-afc8-85379a3fa46c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:48.399578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-35690a68-edcb-4673-afc8-85379a3fa46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:48.432612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e57c6f-fece-497d-9d50-4e11cf5e0095 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:48.438368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f06b6-a5f5-48af-89fd-159c28a1ffd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:48.439158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f06b6-a5f5-48af-89fd-159c28a1ffd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.439967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f06b6-a5f5-48af-89fd-159c28a1ffd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.440255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f06b6-a5f5-48af-89fd-159c28a1ffd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.440977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f06b6-a5f5-48af-89fd-159c28a1ffd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:48.443635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f06b6-a5f5-48af-89fd-159c28a1ffd3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:48.461132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35690a68-edcb-4673-afc8-85379a3fa46c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:48.464737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:48.517540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac6754-8201-45e0-881e-e6a35dbe2fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:48.519216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac6754-8201-45e0-881e-e6a35dbe2fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.519970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac6754-8201-45e0-881e-e6a35dbe2fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.520433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac6754-8201-45e0-881e-e6a35dbe2fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.541010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac6754-8201-45e0-881e-e6a35dbe2fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:48.543351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac6754-8201-45e0-881e-e6a35dbe2fb7 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:48.556001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 938/1862] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:48.569942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 925/1863] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:48 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:48.571452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac6754-8201-45e0-881e-e6a35dbe2fb7 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:48.582201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac6754-8201-45e0-881e-e6a35dbe2fb7 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:48.588732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e239e5-8c9b-4585-861f-8e7544d121d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:48.589301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e239e5-8c9b-4585-861f-8e7544d121d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.589589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e239e5-8c9b-4585-861f-8e7544d121d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.589875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e239e5-8c9b-4585-861f-8e7544d121d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.591051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e239e5-8c9b-4585-861f-8e7544d121d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:48.593138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e239e5-8c9b-4585-861f-8e7544d121d0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:48.603345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 939/1864] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:48.607690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f06b6-a5f5-48af-89fd-159c28a1ffd3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:48.623804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f06b6-a5f5-48af-89fd-159c28a1ffd3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:48.638502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f09a40-fafa-45d3-85c4-ac20a73789b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:48.639530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f09a40-fafa-45d3-85c4-ac20a73789b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.643066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f09a40-fafa-45d3-85c4-ac20a73789b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.643457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f09a40-fafa-45d3-85c4-ac20a73789b6 admin admin] PATH_INFO: `/v3/projects/8923fbaf1c8a48d9a41cf472a492c56d/users/5f55c958b39349fd8cb61fcb4b091862/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.675626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f09a40-fafa-45d3-85c4-ac20a73789b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8923fbaf1c8a48d9a41cf472a492c56d, user_id=5f55c958b39349fd8cb61fcb4b091862, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:48.679205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f09a40-fafa-45d3-85c4-ac20a73789b6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:48.772449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 926/1865] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:48 2022] PUT /identity/v3/projects/8923fbaf1c8a48d9a41cf472a492c56d/users/5f55c958b39349fd8cb61fcb4b091862/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:48.776998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e239e5-8c9b-4585-861f-8e7544d121d0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:48.788914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e239e5-8c9b-4585-861f-8e7544d121d0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:48.852999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa538aba-7bc5-4dba-b65b-d439c3576f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:48.853951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa538aba-7bc5-4dba-b65b-d439c3576f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.854291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa538aba-7bc5-4dba-b65b-d439c3576f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.859405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa538aba-7bc5-4dba-b65b-d439c3576f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.859405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa538aba-7bc5-4dba-b65b-d439c3576f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:48.859405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa538aba-7bc5-4dba-b65b-d439c3576f98 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:48.869225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 940/1866] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:48.873384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f09a40-fafa-45d3-85c4-ac20a73789b6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:48.884763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f09a40-fafa-45d3-85c4-ac20a73789b6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:48.891365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:48.892385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.893355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.894258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] PATH_INFO: `/v3/projects/8923fbaf1c8a48d9a41cf472a492c56d/users/5f55c958b39349fd8cb61fcb4b091862/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.896703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8923fbaf1c8a48d9a41cf472a492c56d, user_id=5f55c958b39349fd8cb61fcb4b091862, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:48.899506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:48.920843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 927/1867] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:48 2022] PUT /identity/v3/projects/8923fbaf1c8a48d9a41cf472a492c56d/users/5f55c958b39349fd8cb61fcb4b091862/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:48.922390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75434c0c-b679-44a9-a107-2ea856776c68 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:48.922766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75434c0c-b679-44a9-a107-2ea856776c68 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:48.923117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75434c0c-b679-44a9-a107-2ea856776c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:48.955023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-75434c0c-b679-44a9-a107-2ea856776c68 None None] MFA Rules not processed for user `5f55c958b39349fd8cb61fcb4b091862`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:48.957269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-75434c0c-b679-44a9-a107-2ea856776c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:49.038565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 941/1868] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:49.052378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:49.062270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:49.088657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75434c0c-b679-44a9-a107-2ea856776c68 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:49.093781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-75434c0c-b679-44a9-a107-2ea856776c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:49.166004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:49.170143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3e17d-d710-4ac1-bf6e-65bfeb01ac48 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:49.177140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:49.178348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:49.178843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:49.179388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:49.181933 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:49.230541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:49.271543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75434c0c-b679-44a9-a107-2ea856776c68 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:49.274704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:49.276469 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:49.283972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:49.284912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:49.285224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:49.285506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] PATH_INFO: `/v3/projects/26accbd88d5243c4af00a5fb5225aabb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:49.287541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26accbd88d5243c4af00a5fb5225aabb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:49.297584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:49.333977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 928/1869] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:49.509749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26accbd88d5243c4af00a5fb5225aabb', 'request_id': 'req-a82838d4-5a79-4654-85ef-e3e2b69fc927'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:49.548735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26accbd88d5243c4af00a5fb5225aabb', 'request_id': 'req-a82838d4-5a79-4654-85ef-e3e2b69fc927'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:49.570447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26accbd88d5243c4af00a5fb5225aabb', 'request_id': 'req-a82838d4-5a79-4654-85ef-e3e2b69fc927'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:49.680355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] The token cache is being invalidate because project 26accbd88d5243c4af00a5fb5225aabb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:49.680355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:49.680957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:49.685688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 942/1870] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:49 2022] DELETE /identity/v3/projects/26accbd88d5243c4af00a5fb5225aabb => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:49.691211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:49.702484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c9b3a-a773-4804-9f9e-d42e0b88cf30 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:49.797335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:49.798643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:49.800853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:49.801415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] PATH_INFO: `/v3/users/ec5079ce2bd54ea59523ff08bc1d8cc6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:49.805487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5079ce2bd54ea59523ff08bc1d8cc6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:49.807714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:50.104113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5079ce2bd54ea59523ff08bc1d8cc6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:50.120354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5079ce2bd54ea59523ff08bc1d8cc6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:50.121275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5079ce2bd54ea59523ff08bc1d8cc6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:50.125815 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5079ce2bd54ea59523ff08bc1d8cc6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:50.245059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-791d0675-6be3-4eb5-abae-299b74ea9d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5079ce2bd54ea59523ff08bc1d8cc6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:50.260443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 929/1871] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:49 2022] DELETE /identity/v3/users/ec5079ce2bd54ea59523ff08bc1d8cc6 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:50.944696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:50.946705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:50.963386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82838d4-5a79-4654-85ef-e3e2b69fc927 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:51.024488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:51.025379 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:51.025662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:51.025938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] PATH_INFO: `/v3/projects/5d2d078062914a7fb9bb653be52ca36d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:51.027102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d2d078062914a7fb9bb653be52ca36d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:51.029018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:51.358592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d2d078062914a7fb9bb653be52ca36d', 'request_id': 'req-787d739a-ceb2-4f1a-9601-202e49e269f0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:51.360678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:51.366218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d2d078062914a7fb9bb653be52ca36d', 'request_id': 'req-787d739a-ceb2-4f1a-9601-202e49e269f0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:51.371971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d2d078062914a7fb9bb653be52ca36d', 'request_id': 'req-787d739a-ceb2-4f1a-9601-202e49e269f0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:51.478781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] The token cache is being invalidate because project 5d2d078062914a7fb9bb653be52ca36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:51.484823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:51.484823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-787d739a-ceb2-4f1a-9601-202e49e269f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:51.484823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 943/1872] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:50 2022] DELETE /identity/v3/projects/5d2d078062914a7fb9bb653be52ca36d => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:51.487013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548c72b-4480-438a-91c7-9cac883c0b7a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:51.487533 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548c72b-4480-438a-91c7-9cac883c0b7a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:51.487620 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548c72b-4480-438a-91c7-9cac883c0b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:51.508300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0548c72b-4480-438a-91c7-9cac883c0b7a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:51.517424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0548c72b-4480-438a-91c7-9cac883c0b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:51.558296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:51.572539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 930/1873] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:51.581202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c398d-50ed-4216-b094-879ff0845d1c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:51.581495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c398d-50ed-4216-b094-879ff0845d1c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:51.581777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c398d-50ed-4216-b094-879ff0845d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:51.602991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9d7c398d-50ed-4216-b094-879ff0845d1c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:51.604198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9d7c398d-50ed-4216-b094-879ff0845d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:51.612574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 944/1874] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:51.616922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548c72b-4480-438a-91c7-9cac883c0b7a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:51.626903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548c72b-4480-438a-91c7-9cac883c0b7a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:51.631612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:51.632281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:51.632569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:51.632850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:51.633616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:51.633929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:51.635540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:51.647717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 931/1875] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:36:51.652106 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c398d-50ed-4216-b094-879ff0845d1c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:51.662274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c398d-50ed-4216-b094-879ff0845d1c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:51.668658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e9cc2-efe9-4dbf-9837-5f872c422384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:51.669358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e9cc2-efe9-4dbf-9837-5f872c422384 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:51.669647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e9cc2-efe9-4dbf-9837-5f872c422384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:51.669925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e9cc2-efe9-4dbf-9837-5f872c422384 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:51.670619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e9cc2-efe9-4dbf-9837-5f872c422384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:51.672523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e9cc2-efe9-4dbf-9837-5f872c422384 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:51.762348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:51.764784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:51.803756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 945/1876] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:51 2022] POST /identity/v3/projects => generated 399 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:51.804654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e9cc2-efe9-4dbf-9837-5f872c422384 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:51.823437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d5564-4280-489c-9ff8-f6311a152ff7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:51.869707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e9cc2-efe9-4dbf-9837-5f872c422384 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:51.875066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f5d87-7708-4c72-98e5-acbaf73d8000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:51.875919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f5d87-7708-4c72-98e5-acbaf73d8000 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:51.876359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f5d87-7708-4c72-98e5-acbaf73d8000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:51.876802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f5d87-7708-4c72-98e5-acbaf73d8000 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:51.876980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:51.877648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:51.877937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:51.878218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:51.878641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f5d87-7708-4c72-98e5-acbaf73d8000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:51.880097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:51.881247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f5d87-7708-4c72-98e5-acbaf73d8000 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:51.955872 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:51.957340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:51.966285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 932/1877] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:51.967087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:51.968402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:51.984321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 946/1878] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:51 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:51.990938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f5d87-7708-4c72-98e5-acbaf73d8000 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:51.998194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55187045-df9a-4058-bb15-ce99c062a2af service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:51.999758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f5d87-7708-4c72-98e5-acbaf73d8000 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.007512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9f1e8-69fc-490b-8cb1-c2b5fdbfc907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.008357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.008492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9f1e8-69fc-490b-8cb1-c2b5fdbfc907 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.008882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9f1e8-69fc-490b-8cb1-c2b5fdbfc907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.009195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9f1e8-69fc-490b-8cb1-c2b5fdbfc907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.009636 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.010099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.010708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9f1e8-69fc-490b-8cb1-c2b5fdbfc907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.010928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] PATH_INFO: `/v3/users/90d4f9a74b0f4a61a7f0e6b57bcff62e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.012568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=90d4f9a74b0f4a61a7f0e6b57bcff62e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.013329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9f1e8-69fc-490b-8cb1-c2b5fdbfc907 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.014763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.028231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 947/1879] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:52.032172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9f1e8-69fc-490b-8cb1-c2b5fdbfc907 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.058779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9f1e8-69fc-490b-8cb1-c2b5fdbfc907 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.063804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a739-3f7f-4500-9d43-5788762b740a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.065320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a739-3f7f-4500-9d43-5788762b740a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.065683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a739-3f7f-4500-9d43-5788762b740a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.066140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5a739-3f7f-4500-9d43-5788762b740a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.067086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5a739-3f7f-4500-9d43-5788762b740a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.069021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5a739-3f7f-4500-9d43-5788762b740a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.080957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 948/1880] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:52.084641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a739-3f7f-4500-9d43-5788762b740a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.096803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5a739-3f7f-4500-9d43-5788762b740a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.101501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11bfd8-9bfa-404a-be58-120afc9286cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.105676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11bfd8-9bfa-404a-be58-120afc9286cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.105676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11bfd8-9bfa-404a-be58-120afc9286cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.105676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11bfd8-9bfa-404a-be58-120afc9286cf admin admin] PATH_INFO: `/v3/projects/d74e31d32bd84fe0a2307ad98db2303c/users/b1a8157fa7e643adbe4b53ba6f853664/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.117698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11bfd8-9bfa-404a-be58-120afc9286cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74e31d32bd84fe0a2307ad98db2303c, user_id=b1a8157fa7e643adbe4b53ba6f853664, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.120321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11bfd8-9bfa-404a-be58-120afc9286cf admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.198961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 949/1881] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:52 2022] PUT /identity/v3/projects/d74e31d32bd84fe0a2307ad98db2303c/users/b1a8157fa7e643adbe4b53ba6f853664/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:52.208152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:52.208152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90d4f9a74b0f4a61a7f0e6b57bcff62e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:52.209607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385c63-98c5-41e1-a048-366bd1ee2c03 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.209906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385c63-98c5-41e1-a048-366bd1ee2c03 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.210211 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385c63-98c5-41e1-a048-366bd1ee2c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.220464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90d4f9a74b0f4a61a7f0e6b57bcff62e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:52.221289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90d4f9a74b0f4a61a7f0e6b57bcff62e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:52.221950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90d4f9a74b0f4a61a7f0e6b57bcff62e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:52.235565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dd385c63-98c5-41e1-a048-366bd1ee2c03 None None] MFA Rules not processed for user `b1a8157fa7e643adbe4b53ba6f853664`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:52.237086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dd385c63-98c5-41e1-a048-366bd1ee2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:52.289576 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:52.309721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 950/1882] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:52.313989 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385c63-98c5-41e1-a048-366bd1ee2c03 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.315800 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dd385c63-98c5-41e1-a048-366bd1ee2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:52.318282 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4537fdb5-849d-414d-9c44-8f6c943122ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90d4f9a74b0f4a61a7f0e6b57bcff62e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:52.331618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 933/1883] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:51 2022] DELETE /identity/v3/users/90d4f9a74b0f4a61a7f0e6b57bcff62e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:52.335829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e9e7-3ec0-4903-95eb-da2168753fad None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.336120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e9e7-3ec0-4903-95eb-da2168753fad None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.336429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e9e7-3ec0-4903-95eb-da2168753fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.355077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a131e9e7-3ec0-4903-95eb-da2168753fad None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:52.356864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a131e9e7-3ec0-4903-95eb-da2168753fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:52.380025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385c63-98c5-41e1-a048-366bd1ee2c03 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.416225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 934/1884] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:52.426492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131e9e7-3ec0-4903-95eb-da2168753fad None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.427930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a131e9e7-3ec0-4903-95eb-da2168753fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:52.437708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e850928-ca28-4597-9eda-85269ecd09fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.438384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e850928-ca28-4597-9eda-85269ecd09fc admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.438803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e850928-ca28-4597-9eda-85269ecd09fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.440486 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e850928-ca28-4597-9eda-85269ecd09fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.441758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e850928-ca28-4597-9eda-85269ecd09fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.448712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e850928-ca28-4597-9eda-85269ecd09fc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.488266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131e9e7-3ec0-4903-95eb-da2168753fad None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.491912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b58a24-93a9-4a53-8d74-13db75974f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.492552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b58a24-93a9-4a53-8d74-13db75974f9c service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.492849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b58a24-93a9-4a53-8d74-13db75974f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.493126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b58a24-93a9-4a53-8d74-13db75974f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.502543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b58a24-93a9-4a53-8d74-13db75974f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.503963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b58a24-93a9-4a53-8d74-13db75974f9c service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.513194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 935/1885] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:52.556229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 951/1886] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:52 2022] POST /identity/v3/projects => generated 401 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:52.560504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b58a24-93a9-4a53-8d74-13db75974f9c service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.571326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b58a24-93a9-4a53-8d74-13db75974f9c service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.636603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e0642-860c-42b8-a952-a567ba572607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.637296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0642-860c-42b8-a952-a567ba572607 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.637697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0642-860c-42b8-a952-a567ba572607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.637983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0642-860c-42b8-a952-a567ba572607 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.638683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e0642-860c-42b8-a952-a567ba572607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.640630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e0642-860c-42b8-a952-a567ba572607 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.650290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:52.719390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 936/1887] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:52 2022] POST /identity/v3/users => generated 307 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:52.724761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e850928-ca28-4597-9eda-85269ecd09fc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.740195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e850928-ca28-4597-9eda-85269ecd09fc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.745617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7d8f5-65b8-4f21-82d3-4ae23738c317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.749232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7d8f5-65b8-4f21-82d3-4ae23738c317 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.749232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7d8f5-65b8-4f21-82d3-4ae23738c317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.749232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7d8f5-65b8-4f21-82d3-4ae23738c317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.749232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7d8f5-65b8-4f21-82d3-4ae23738c317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.749685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7d8f5-65b8-4f21-82d3-4ae23738c317 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.762352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 952/1888] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:36:52.767840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e0642-860c-42b8-a952-a567ba572607 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.781594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e0642-860c-42b8-a952-a567ba572607 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.786879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.787814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.788094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.788370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] PATH_INFO: `/v3/projects/8aae60b1b83b45bf9aa799efc1dbd1fc/users/bbbfceaa043842dc8ce173e47eebf2dc/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.804357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aae60b1b83b45bf9aa799efc1dbd1fc, user_id=bbbfceaa043842dc8ce173e47eebf2dc, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.808398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.914056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 937/1889] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:52 2022] PUT /identity/v3/projects/8aae60b1b83b45bf9aa799efc1dbd1fc/users/bbbfceaa043842dc8ce173e47eebf2dc/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:52.918005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7d8f5-65b8-4f21-82d3-4ae23738c317 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.928484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7d8f5-65b8-4f21-82d3-4ae23738c317 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:52.983571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba9bb0-f496-4d06-9c6e-7949b44a0a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:52.984241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9bb0-f496-4d06-9c6e-7949b44a0a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:52.984527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9bb0-f496-4d06-9c6e-7949b44a0a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:52.985026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba9bb0-f496-4d06-9c6e-7949b44a0a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:52.985593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba9bb0-f496-4d06-9c6e-7949b44a0a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:52.987743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ba9bb0-f496-4d06-9c6e-7949b44a0a72 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:52.990315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:52.995767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:53.000753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 953/1890] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:53.006160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba9bb0-f496-4d06-9c6e-7949b44a0a72 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:53.011783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:53.012124 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f3137-e51f-4b24-8adc-c1fab0a15d74 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:53.017426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:53.017646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba9bb0-f496-4d06-9c6e-7949b44a0a72 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:53.018545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:53.018834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:53.019139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] PATH_INFO: `/v3/projects/ec50599e2b7b4e958c0cf261b17eed02` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:53.020594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec50599e2b7b4e958c0cf261b17eed02)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:53.022670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:53.023284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9e356-a34e-4b45-87d2-9dcb371b2c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:53.024546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9e356-a34e-4b45-87d2-9dcb371b2c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:53.024928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9e356-a34e-4b45-87d2-9dcb371b2c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:53.025285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9e356-a34e-4b45-87d2-9dcb371b2c01 admin admin] PATH_INFO: `/v3/projects/8aae60b1b83b45bf9aa799efc1dbd1fc/users/bbbfceaa043842dc8ce173e47eebf2dc/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:53.027468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9e356-a34e-4b45-87d2-9dcb371b2c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aae60b1b83b45bf9aa799efc1dbd1fc, user_id=bbbfceaa043842dc8ce173e47eebf2dc, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:53.030104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9e356-a34e-4b45-87d2-9dcb371b2c01 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:53.040919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 954/1891] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:53 2022] PUT /identity/v3/projects/8aae60b1b83b45bf9aa799efc1dbd1fc/users/bbbfceaa043842dc8ce173e47eebf2dc/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:53.053296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc96b3-8c69-4254-81b5-703fac9402c6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:53.053596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc96b3-8c69-4254-81b5-703fac9402c6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:53.053882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc96b3-8c69-4254-81b5-703fac9402c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:53.085469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a0fc96b3-8c69-4254-81b5-703fac9402c6 None None] MFA Rules not processed for user `bbbfceaa043842dc8ce173e47eebf2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:53.088153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a0fc96b3-8c69-4254-81b5-703fac9402c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:53.160662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 955/1892] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:53 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:53.171627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc96b3-8c69-4254-81b5-703fac9402c6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:53.171777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec50599e2b7b4e958c0cf261b17eed02', 'request_id': 'req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.174109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a0fc96b3-8c69-4254-81b5-703fac9402c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:53.181045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec50599e2b7b4e958c0cf261b17eed02', 'request_id': 'req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.190137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec50599e2b7b4e958c0cf261b17eed02', 'request_id': 'req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.235739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc96b3-8c69-4254-81b5-703fac9402c6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:53.284119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252ca32-a166-4a5c-8912-1556a08d2145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:53.285217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252ca32-a166-4a5c-8912-1556a08d2145 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:53.285501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252ca32-a166-4a5c-8912-1556a08d2145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:53.285783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252ca32-a166-4a5c-8912-1556a08d2145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:53.318438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] The token cache is being invalidate because project ec50599e2b7b4e958c0cf261b17eed02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:53.318981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.319738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.321870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 938/1893] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:52 2022] DELETE /identity/v3/projects/ec50599e2b7b4e958c0cf261b17eed02 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:53.325778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:53.327221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:53.343704 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:53.354998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252ca32-a166-4a5c-8912-1556a08d2145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:53.356444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252ca32-a166-4a5c-8912-1556a08d2145 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:53.366356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 956/1894] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:53 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:53.391875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd4afc-f03c-4aa2-8616-1c6f04d2c6f1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:53.397318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:53.398197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:53.398483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:53.398764 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] PATH_INFO: `/v3/users/944c9bdc04bb44429d62857fa85601da` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:53.399995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944c9bdc04bb44429d62857fa85601da)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:53.402025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:53.637863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252ca32-a166-4a5c-8912-1556a08d2145 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:53.647146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252ca32-a166-4a5c-8912-1556a08d2145 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:53.651149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:53.651788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:53.652074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:53.652352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:53.654985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:53.700016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944c9bdc04bb44429d62857fa85601da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.708924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:INSERT=3,keystone:SELECT=82,keystone:DELETE=15 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:53.730883 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944c9bdc04bb44429d62857fa85601da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.731643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944c9bdc04bb44429d62857fa85601da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.732398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944c9bdc04bb44429d62857fa85601da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.737601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:53.739836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:53.753693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 957/1895] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:53.754586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:53.756004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:53.769495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a6b20-ee09-4e82-9ec3-e317746a0e3a service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:53.773437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d308b37-d53f-4fae-9d21-39d374b91b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:53.774559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308b37-d53f-4fae-9d21-39d374b91b37 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:53.774859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308b37-d53f-4fae-9d21-39d374b91b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:53.775139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d308b37-d53f-4fae-9d21-39d374b91b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:53.776062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944c9bdc04bb44429d62857fa85601da'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:53.777371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d308b37-d53f-4fae-9d21-39d374b91b37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:53.797681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 939/1896] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:53 2022] DELETE /identity/v3/users/944c9bdc04bb44429d62857fa85601da => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:53.865044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d308b37-d53f-4fae-9d21-39d374b91b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:53.866940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d308b37-d53f-4fae-9d21-39d374b91b37 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:53.881063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 958/1897] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:54.758946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:54.760462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:54.837256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e83d7-ccc4-4204-8010-9a51f16143e5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:54.841926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:54.842868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:54.843153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:54.843429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] PATH_INFO: `/v3/projects/93ba0eb6469d4d779436605f40642983` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:54.844604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ba0eb6469d4d779436605f40642983)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:54.847221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:54.980939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ba0eb6469d4d779436605f40642983', 'request_id': 'req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:54.989060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ba0eb6469d4d779436605f40642983', 'request_id': 'req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:54.995511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ba0eb6469d4d779436605f40642983', 'request_id': 'req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:55.061183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] The token cache is being invalidate because project 93ba0eb6469d4d779436605f40642983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:36:55.061483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:55.062114 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f7167ad7-629c-4ef5-b6f1-20b8615f05d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:55.063665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 940/1898] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:36:54 2022] DELETE /identity/v3/projects/93ba0eb6469d4d779436605f40642983 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:55.068789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b6816-17a9-4110-a0d5-ca1b73efb203 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.069300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b6816-17a9-4110-a0d5-ca1b73efb203 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.069402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b6816-17a9-4110-a0d5-ca1b73efb203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.087449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5c6b6816-17a9-4110-a0d5-ca1b73efb203 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:55.088633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5c6b6816-17a9-4110-a0d5-ca1b73efb203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:55.159584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 959/1899] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:55.167314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3435cd8-0dd8-409f-ae9a-b1e4d20316b3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.167602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3435cd8-0dd8-409f-ae9a-b1e4d20316b3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.167878 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3435cd8-0dd8-409f-ae9a-b1e4d20316b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.188281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b3435cd8-0dd8-409f-ae9a-b1e4d20316b3 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:55.189558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b3435cd8-0dd8-409f-ae9a-b1e4d20316b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:55.197639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 941/1900] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:55.201913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b6816-17a9-4110-a0d5-ca1b73efb203 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:55.221583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b6816-17a9-4110-a0d5-ca1b73efb203 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:55.221583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:55.221583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.221583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.221914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.221914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:36:55.221914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:55.221914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:55.231256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:55.234870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 960/1901] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:36:55.239787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3435cd8-0dd8-409f-ae9a-b1e4d20316b3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:55.250660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3435cd8-0dd8-409f-ae9a-b1e4d20316b3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:55.255733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec659bd-cebf-47fa-ab9d-f97b676c038b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:55.256402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec659bd-cebf-47fa-ab9d-f97b676c038b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.256687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec659bd-cebf-47fa-ab9d-f97b676c038b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.256965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec659bd-cebf-47fa-ab9d-f97b676c038b admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.257697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec659bd-cebf-47fa-ab9d-f97b676c038b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:55.259714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec659bd-cebf-47fa-ab9d-f97b676c038b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:55.371862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 942/1902] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:36:55 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:36:55.376230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:55.386349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd8206-9edd-4c0f-a6dd-4301b42667f0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:55.445155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c3aa8-d22a-4074-b6ee-b5399bb26cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:55.445792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c3aa8-d22a-4074-b6ee-b5399bb26cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.446099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c3aa8-d22a-4074-b6ee-b5399bb26cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.446373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c3aa8-d22a-4074-b6ee-b5399bb26cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.447063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c3aa8-d22a-4074-b6ee-b5399bb26cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:55.449003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c3aa8-d22a-4074-b6ee-b5399bb26cc7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:55.596684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 961/1903] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:36:55 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:36:55.599539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec659bd-cebf-47fa-ab9d-f97b676c038b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:55.617434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec659bd-cebf-47fa-ab9d-f97b676c038b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:55.622630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67b26c-dd8a-49f9-bcc3-ab225ac667a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:55.623306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67b26c-dd8a-49f9-bcc3-ab225ac667a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.624023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67b26c-dd8a-49f9-bcc3-ab225ac667a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.624023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67b26c-dd8a-49f9-bcc3-ab225ac667a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.625044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67b26c-dd8a-49f9-bcc3-ab225ac667a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:55.626438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67b26c-dd8a-49f9-bcc3-ab225ac667a8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:55.638498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 943/1904] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:55.643478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c3aa8-d22a-4074-b6ee-b5399bb26cc7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:55.655598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c3aa8-d22a-4074-b6ee-b5399bb26cc7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:55.662694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0254f2-9974-42cb-998f-34fdba2a834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:55.664680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0254f2-9974-42cb-998f-34fdba2a834e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.665576 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0254f2-9974-42cb-998f-34fdba2a834e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.669375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0254f2-9974-42cb-998f-34fdba2a834e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.669375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0254f2-9974-42cb-998f-34fdba2a834e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:55.669375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0254f2-9974-42cb-998f-34fdba2a834e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:55.678960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 962/1905] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:36:55.683506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67b26c-dd8a-49f9-bcc3-ab225ac667a8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:55.697289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67b26c-dd8a-49f9-bcc3-ab225ac667a8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:55.702884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:55.703781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.704065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.704343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] PATH_INFO: `/v3/projects/c877d4e75dbc4bdbb9fd24cbc78c75b0/users/9cbe6758fa024de5af49b49e340618aa/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.719743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c877d4e75dbc4bdbb9fd24cbc78c75b0, user_id=9cbe6758fa024de5af49b49e340618aa, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:55.722459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:55.851548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:55.857796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 944/1906] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:36:55 2022] PUT /identity/v3/projects/c877d4e75dbc4bdbb9fd24cbc78c75b0/users/9cbe6758fa024de5af49b49e340618aa/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:55.869439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c58b2e-ff19-41f3-90bc-53a6e6ed7c25 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:55.869734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c58b2e-ff19-41f3-90bc-53a6e6ed7c25 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:55.870060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c58b2e-ff19-41f3-90bc-53a6e6ed7c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:55.896793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-99c58b2e-ff19-41f3-90bc-53a6e6ed7c25 None None] MFA Rules not processed for user `9cbe6758fa024de5af49b49e340618aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:55.898017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-99c58b2e-ff19-41f3-90bc-53a6e6ed7c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:55.974199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 963/1907] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:55.985684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:55.987670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:56.042740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf10f5-5bdf-4503-ada0-f898bc97e71b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:56.046532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47589bf9e7c4acba200979e3a91a6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:56.047181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:56.047466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:56.047746 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:56.067710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:56.069639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:56.079961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 945/1908] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:36:58.874960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c58b2e-ff19-41f3-90bc-53a6e6ed7c25 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:58.877456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-99c58b2e-ff19-41f3-90bc-53a6e6ed7c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:58.954985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c58b2e-ff19-41f3-90bc-53a6e6ed7c25 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:58.959563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:58.960439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:58.960726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:58.961008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] PATH_INFO: `/v3/users/36e7565ddb574fbc874d46fb80b881a7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:58.962264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e7565ddb574fbc874d46fb80b881a7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:58.964235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:58.979247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:59.197978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e7565ddb574fbc874d46fb80b881a7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.204130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:59.205806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:59.207624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e7565ddb574fbc874d46fb80b881a7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.225908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e7565ddb574fbc874d46fb80b881a7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.227618 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e7565ddb574fbc874d46fb80b881a7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.230402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bfef9-8594-4f7f-98fb-a6fd738a62fe service glance] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:59.238476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:59.239356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:59.239641 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:59.239919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] PATH_INFO: `/v3/users/b1a8157fa7e643adbe4b53ba6f853664` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:59.241317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a8157fa7e643adbe4b53ba6f853664)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:59.243530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:59.294657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-81f8dcfe-6903-4133-ac89-43d6a6743203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e7565ddb574fbc874d46fb80b881a7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.301935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 964/1909] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:58 2022] DELETE /identity/v3/users/36e7565ddb574fbc874d46fb80b881a7 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:36:59.306638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5535056-dc8b-4e06-9f22-58f32ac02bbc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:59.306934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5535056-dc8b-4e06-9f22-58f32ac02bbc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:59.307217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5535056-dc8b-4e06-9f22-58f32ac02bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:59.327292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e5535056-dc8b-4e06-9f22-58f32ac02bbc None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:59.328688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e5535056-dc8b-4e06-9f22-58f32ac02bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:59.398387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 965/1910] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:36:59.408212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5535056-dc8b-4e06-9f22-58f32ac02bbc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:59.409668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e5535056-dc8b-4e06-9f22-58f32ac02bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:59.435165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a8157fa7e643adbe4b53ba6f853664'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.449350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a8157fa7e643adbe4b53ba6f853664'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.450108 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a8157fa7e643adbe4b53ba6f853664'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.450740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a8157fa7e643adbe4b53ba6f853664'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.477654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5535056-dc8b-4e06-9f22-58f32ac02bbc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:59.481716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0097d-5b68-42ec-bc4f-4384b7c6673f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:59.482477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0097d-5b68-42ec-bc4f-4384b7c6673f service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:59.482645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0097d-5b68-42ec-bc4f-4384b7c6673f service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:59.482929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0097d-5b68-42ec-bc4f-4384b7c6673f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:59.484810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e0f0097d-5b68-42ec-bc4f-4384b7c6673f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:59.500135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0097d-5b68-42ec-bc4f-4384b7c6673f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:59.501655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0097d-5b68-42ec-bc4f-4384b7c6673f service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:59.511387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 966/1911] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:36:59.528829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a8157fa7e643adbe4b53ba6f853664'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:36:59.542301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 946/1912] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:36:59 2022] DELETE /identity/v3/users/b1a8157fa7e643adbe4b53ba6f853664 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:36:59.546873 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e6545-887f-4f4f-af32-cdd1de4226da None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:59.547167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e6545-887f-4f4f-af32-cdd1de4226da None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:59.547449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e6545-887f-4f4f-af32-cdd1de4226da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:59.568025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6a7e6545-887f-4f4f-af32-cdd1de4226da None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:36:59.569277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6a7e6545-887f-4f4f-af32-cdd1de4226da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:59.598775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:36:59.635096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 967/1913] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:36:59.645384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:36:59.646822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:36:59.710933 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e5589-50a4-4022-ab48-d08bc3d35fc3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:36:59.717274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e587f5-4731-4c06-a0cf-c56fecb9a85a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:36:59.717965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e587f5-4731-4c06-a0cf-c56fecb9a85a service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:36:59.718249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e587f5-4731-4c06-a0cf-c56fecb9a85a service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:36:59.718551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e587f5-4731-4c06-a0cf-c56fecb9a85a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:36:59.729588 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e587f5-4731-4c06-a0cf-c56fecb9a85a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:36:59.730996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e587f5-4731-4c06-a0cf-c56fecb9a85a service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:36:59.742106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 947/1914] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:00.090258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a26f57-85a6-4c45-80e1-5da82c0fb184 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:00.097536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a26f57-85a6-4c45-80e1-5da82c0fb184 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:00.097536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a26f57-85a6-4c45-80e1-5da82c0fb184 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:00.099107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 968/1915] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:37:00 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 03:37:00.112569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc5b6f-dc4d-467e-8e90-d11d66f20c4e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:00.112858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc5b6f-dc4d-467e-8e90-d11d66f20c4e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:00.113178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc5b6f-dc4d-467e-8e90-d11d66f20c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:00.134631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dadc5b6f-dc4d-467e-8e90-d11d66f20c4e None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:00.135835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dadc5b6f-dc4d-467e-8e90-d11d66f20c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:00.146286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:00.204408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 948/1916] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:37:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:00.218045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a26f57-85a6-4c45-80e1-5da82c0fb184 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:00.219976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-75a26f57-85a6-4c45-80e1-5da82c0fb184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:00.233627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc5b6f-dc4d-467e-8e90-d11d66f20c4e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:00.235469 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dadc5b6f-dc4d-467e-8e90-d11d66f20c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:00.237532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a26f57-85a6-4c45-80e1-5da82c0fb184 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:00.243354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:00.244586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:00.245043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:00.245545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:00.255775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc5b6f-dc4d-467e-8e90-d11d66f20c4e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:00.261056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:00.262491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:00.264143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:00.265507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:00.265868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:00.266246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] PATH_INFO: `/v3/projects/d74e31d32bd84fe0a2307ad98db2303c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:00.267650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d74e31d32bd84fe0a2307ad98db2303c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:00.275570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:00.276844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 969/1917] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:00.294537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:00.296234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:00.316225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ad944-4b9c-4df2-bb59-ff3dd45821e5 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:00.321491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:00.322709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:00.322920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:00.323212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] PATH_INFO: `/v3/users/46e48b861bc2498c9670de030c783f54` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:00.325378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e48b861bc2498c9670de030c783f54)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:00.327378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:00.399440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd74e31d32bd84fe0a2307ad98db2303c', 'request_id': 'req-796666a7-90df-4b73-a67d-b8ac33ee2406'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.406220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd74e31d32bd84fe0a2307ad98db2303c', 'request_id': 'req-796666a7-90df-4b73-a67d-b8ac33ee2406'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.411896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd74e31d32bd84fe0a2307ad98db2303c', 'request_id': 'req-796666a7-90df-4b73-a67d-b8ac33ee2406'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.518046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] The token cache is being invalidate because project d74e31d32bd84fe0a2307ad98db2303c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:00.518450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.519744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.521396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 949/1918] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:00 2022] DELETE /identity/v3/projects/d74e31d32bd84fe0a2307ad98db2303c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:00.522164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:00.523474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:00.576415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e48b861bc2498c9670de030c783f54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.584135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e48b861bc2498c9670de030c783f54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.597340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e48b861bc2498c9670de030c783f54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.597340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e48b861bc2498c9670de030c783f54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.597520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796666a7-90df-4b73-a67d-b8ac33ee2406 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:00.606036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:00.606946 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:00.607227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:00.607541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] PATH_INFO: `/v3/projects/e2b15be296924b19b3d68cfc802b4174` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:00.608772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b15be296924b19b3d68cfc802b4174)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:00.610776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:00.689975 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e48b861bc2498c9670de030c783f54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.696749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 970/1919] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:00 2022] DELETE /identity/v3/users/46e48b861bc2498c9670de030c783f54 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:00.700886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:00.700886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:00.762512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:00.771695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c9adc-47c9-4ee9-a25f-0e9794468b8c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:00.773901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b15be296924b19b3d68cfc802b4174', 'request_id': 'req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.780891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b15be296924b19b3d68cfc802b4174', 'request_id': 'req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.789806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b15be296924b19b3d68cfc802b4174', 'request_id': 'req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.817997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:00.846126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c62521-f374-4419-952b-b754da27895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:00.847565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c62521-f374-4419-952b-b754da27895c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:00.847847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c62521-f374-4419-952b-b754da27895c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:00.848127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c62521-f374-4419-952b-b754da27895c admin admin] PATH_INFO: `/v3/users/bbbfceaa043842dc8ce173e47eebf2dc` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:00.849459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c62521-f374-4419-952b-b754da27895c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbbfceaa043842dc8ce173e47eebf2dc)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:00.851404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c62521-f374-4419-952b-b754da27895c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:00.861628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] The token cache is being invalidate because project e2b15be296924b19b3d68cfc802b4174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:00.862675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.863240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:00.868187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 950/1920] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:00 2022] DELETE /identity/v3/projects/e2b15be296924b19b3d68cfc802b4174 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:00.869181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:00.870617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:00.885594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb3c76-a62a-4eb1-bfe3-9bbf3f6e51e9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:00.890534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:00.891409 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:00.891693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:00.891971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] PATH_INFO: `/v3/users/83b9d86f338c42fab588f1af709336fc` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:00.893211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b9d86f338c42fab588f1af709336fc)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:00.895851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:01.024535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33c62521-f374-4419-952b-b754da27895c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbbfceaa043842dc8ce173e47eebf2dc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.031228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33c62521-f374-4419-952b-b754da27895c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbbfceaa043842dc8ce173e47eebf2dc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.043250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33c62521-f374-4419-952b-b754da27895c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbbfceaa043842dc8ce173e47eebf2dc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.044014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33c62521-f374-4419-952b-b754da27895c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbbfceaa043842dc8ce173e47eebf2dc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.092381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b9d86f338c42fab588f1af709336fc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.104191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b9d86f338c42fab588f1af709336fc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.104893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b9d86f338c42fab588f1af709336fc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.105660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b9d86f338c42fab588f1af709336fc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.129791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-33c62521-f374-4419-952b-b754da27895c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbbfceaa043842dc8ce173e47eebf2dc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.139692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 971/1921] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:00 2022] DELETE /identity/v3/users/bbbfceaa043842dc8ce173e47eebf2dc => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:01.140640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c62521-f374-4419-952b-b754da27895c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:01.142013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-33c62521-f374-4419-952b-b754da27895c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:01.215893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2d80a2f4-7e77-4201-a7d5-8729f1668dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b9d86f338c42fab588f1af709336fc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.222552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c62521-f374-4419-952b-b754da27895c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:01.227746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:01.228616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:01.228913 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:01.229190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] PATH_INFO: `/v3/projects/fc3ad7416d27411cb8ddbce0a3da18eb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:01.230635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 951/1922] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:00 2022] DELETE /identity/v3/users/83b9d86f338c42fab588f1af709336fc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:01.230751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3ad7416d27411cb8ddbce0a3da18eb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:01.232416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:01.349957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'request_id': 'req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.356500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'request_id': 'req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.363132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3ad7416d27411cb8ddbce0a3da18eb', 'request_id': 'req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.456566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] The token cache is being invalidate because project fc3ad7416d27411cb8ddbce0a3da18eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:01.460725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.460725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4fd1b646-9bcb-47df-b455-c9ac5649a2d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.460725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 972/1923] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:01 2022] DELETE /identity/v3/projects/fc3ad7416d27411cb8ddbce0a3da18eb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:01.466945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929ef70-e868-426c-b9eb-679068186b19 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:01.467244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929ef70-e868-426c-b9eb-679068186b19 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:01.467538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929ef70-e868-426c-b9eb-679068186b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:01.485900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d929ef70-e868-426c-b9eb-679068186b19 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:01.487932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d929ef70-e868-426c-b9eb-679068186b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:01.549457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 952/1924] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:01.558345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f73c6-e468-4301-8fd6-52bf4bd09f31 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:01.558642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f73c6-e468-4301-8fd6-52bf4bd09f31 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:01.558920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f73c6-e468-4301-8fd6-52bf4bd09f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:01.579052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-562f73c6-e468-4301-8fd6-52bf4bd09f31 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:01.582521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-562f73c6-e468-4301-8fd6-52bf4bd09f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:01.583626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929ef70-e868-426c-b9eb-679068186b19 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:01.585047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d929ef70-e868-426c-b9eb-679068186b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:01.589772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 973/1925] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:01.594185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f73c6-e468-4301-8fd6-52bf4bd09f31 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:01.604435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f73c6-e468-4301-8fd6-52bf4bd09f31 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:01.609343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:01.610007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:01.610294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:01.610568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:01.610776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929ef70-e868-426c-b9eb-679068186b19 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:01.611316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:01.611593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:01.614012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:01.616557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:01.617614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:01.617967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:01.618328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] PATH_INFO: `/v3/projects/8aae60b1b83b45bf9aa799efc1dbd1fc` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:01.619681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aae60b1b83b45bf9aa799efc1dbd1fc)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:01.621807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:01.630684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 974/1926] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:01.634805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:01.646260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b237d18-7265-4d71-895e-3449bec97b13 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:01.651454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:01.652208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:01.652492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:01.652766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:01.653541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:01.655598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:01.683911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:01.761016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:01.782673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aae60b1b83b45bf9aa799efc1dbd1fc', 'request_id': 'req-4ac809cf-dfba-43d2-a267-23778fd321af'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.789919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 975/1927] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:01 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:01.791157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aae60b1b83b45bf9aa799efc1dbd1fc', 'request_id': 'req-4ac809cf-dfba-43d2-a267-23778fd321af'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.791285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:01.792979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:01.799056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aae60b1b83b45bf9aa799efc1dbd1fc', 'request_id': 'req-4ac809cf-dfba-43d2-a267-23778fd321af'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.864899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bded6-a0c2-4d92-9450-3ff86885d27a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:01.869661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:01.872164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:01.872164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:01.872164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] PATH_INFO: `/v3/projects/58b1765f61e54a85ae5b3d15d133c9ed` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:01.872484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58b1765f61e54a85ae5b3d15d133c9ed)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:01.874390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:01.880303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] The token cache is being invalidate because project 8aae60b1b83b45bf9aa799efc1dbd1fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:01.880694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.881508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:01.883281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 953/1928] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:01 2022] DELETE /identity/v3/projects/8aae60b1b83b45bf9aa799efc1dbd1fc => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:01.884218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:01.885588 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:01.953873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac809cf-dfba-43d2-a267-23778fd321af admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:01.959112 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c452fd-afca-4cb7-9dec-fb92f0dcbf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:01.959793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c452fd-afca-4cb7-9dec-fb92f0dcbf31 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:01.960084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c452fd-afca-4cb7-9dec-fb92f0dcbf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:01.960365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c452fd-afca-4cb7-9dec-fb92f0dcbf31 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:01.961059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c452fd-afca-4cb7-9dec-fb92f0dcbf31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:01.963211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c452fd-afca-4cb7-9dec-fb92f0dcbf31 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:01.993977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58b1765f61e54a85ae5b3d15d133c9ed', 'request_id': 'req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.000807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58b1765f61e54a85ae5b3d15d133c9ed', 'request_id': 'req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.006405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58b1765f61e54a85ae5b3d15d133c9ed', 'request_id': 'req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.027307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 954/1929] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:01 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:02.028631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9659e8-8ae0-41c8-b756-5929bd522e42 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.028924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9659e8-8ae0-41c8-b756-5929bd522e42 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.029237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9659e8-8ae0-41c8-b756-5929bd522e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.048740 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1f9659e8-8ae0-41c8-b756-5929bd522e42 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:02.050525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1f9659e8-8ae0-41c8-b756-5929bd522e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:02.095484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] The token cache is being invalidate because project 58b1765f61e54a85ae5b3d15d133c9ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:02.095785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.096580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.098837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 976/1930] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:01 2022] DELETE /identity/v3/projects/58b1765f61e54a85ae5b3d15d133c9ed => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:02.099677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.101082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:02.117260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 955/1931] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:02.117260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9659e8-8ae0-41c8-b756-5929bd522e42 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.117260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1f9659e8-8ae0-41c8-b756-5929bd522e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:02.127248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9659e8-8ae0-41c8-b756-5929bd522e42 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.131988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.132859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.133167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.140054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] PATH_INFO: `/v3/users/5f55c958b39349fd8cb61fcb4b091862` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.140054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f55c958b39349fd8cb61fcb4b091862)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.140054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.171227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c69f-fdc9-4f80-95a8-b33e67d1e349 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.176107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0854c9d-cfa2-4a19-ac4f-c2d81c8a9c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.176769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0854c9d-cfa2-4a19-ac4f-c2d81c8a9c80 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.177055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0854c9d-cfa2-4a19-ac4f-c2d81c8a9c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.177365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0854c9d-cfa2-4a19-ac4f-c2d81c8a9c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.178089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0854c9d-cfa2-4a19-ac4f-c2d81c8a9c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.180080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0854c9d-cfa2-4a19-ac4f-c2d81c8a9c80 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.182637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:02.192178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 977/1932] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:02.193345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374e742-2d91-4e6c-aaa2-4d5a6fccb9fa None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.193847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374e742-2d91-4e6c-aaa2-4d5a6fccb9fa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.194119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374e742-2d91-4e6c-aaa2-4d5a6fccb9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.216049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4374e742-2d91-4e6c-aaa2-4d5a6fccb9fa None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:02.217269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4374e742-2d91-4e6c-aaa2-4d5a6fccb9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:02.231839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 978/1933] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:02.231839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374e742-2d91-4e6c-aaa2-4d5a6fccb9fa None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.235973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374e742-2d91-4e6c-aaa2-4d5a6fccb9fa None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.240848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a2814-ffb7-4730-bc80-15872e62449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.241513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a2814-ffb7-4730-bc80-15872e62449b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.241797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a2814-ffb7-4730-bc80-15872e62449b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.242075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a2814-ffb7-4730-bc80-15872e62449b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.242793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a2814-ffb7-4730-bc80-15872e62449b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.244368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a2814-ffb7-4730-bc80-15872e62449b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.255272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 979/1934] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:02.256150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a2814-ffb7-4730-bc80-15872e62449b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.265776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a2814-ffb7-4730-bc80-15872e62449b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.271144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.271783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.272067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.272349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.273062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:02.273406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.274985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.288349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 980/1935] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:02.289259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.300288 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba406aa-1600-4074-bb7c-19faee3ac6d7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.305520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.306511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.306803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.307082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] PATH_INFO: `/v3/projects/2849d47d98844282a331ba9f5abc0ed9/users/82a0db16d583443cb30058d3ef774103/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.321685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2849d47d98844282a331ba9f5abc0ed9, user_id=82a0db16d583443cb30058d3ef774103, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.324754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.337338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:02.338869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f55c958b39349fd8cb61fcb4b091862'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.351365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f55c958b39349fd8cb61fcb4b091862'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.351996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f55c958b39349fd8cb61fcb4b091862'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.352622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f55c958b39349fd8cb61fcb4b091862'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.392273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 981/1936] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:02 2022] PUT /identity/v3/projects/2849d47d98844282a331ba9f5abc0ed9/users/82a0db16d583443cb30058d3ef774103/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:02.393270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.395343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:02.444375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0e6bf165-6777-44f4-8457-c52b966c7365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f55c958b39349fd8cb61fcb4b091862'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:02.458360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 956/1937] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:02 2022] DELETE /identity/v3/users/5f55c958b39349fd8cb61fcb4b091862 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:02.461959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eb7ac-0ba9-44a8-a555-fc2c62f913e5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.461959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eb7ac-0ba9-44a8-a555-fc2c62f913e5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.462188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df8acb-914d-4bc7-a4df-a7026892910d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.462291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057eb7ac-0ba9-44a8-a555-fc2c62f913e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.491807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-057eb7ac-0ba9-44a8-a555-fc2c62f913e5 None None] MFA Rules not processed for user `82a0db16d583443cb30058d3ef774103`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:02.493303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-057eb7ac-0ba9-44a8-a555-fc2c62f913e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:02.523290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c47e4-a617-471b-ba37-233d741b1a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.523966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c47e4-a617-471b-ba37-233d741b1a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.524256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c47e4-a617-471b-ba37-233d741b1a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.524638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c47e4-a617-471b-ba37-233d741b1a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.525253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c47e4-a617-471b-ba37-233d741b1a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.527459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c47e4-a617-471b-ba37-233d741b1a56 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.574105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 957/1938] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:02.577898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057eb7ac-0ba9-44a8-a555-fc2c62f913e5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.579248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-057eb7ac-0ba9-44a8-a555-fc2c62f913e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:02.592383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057eb7ac-0ba9-44a8-a555-fc2c62f913e5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.597690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fc125-5b32-424d-bef0-a7dd8d2e79ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.598385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fc125-5b32-424d-bef0-a7dd8d2e79ed admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.598766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fc125-5b32-424d-bef0-a7dd8d2e79ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.599052 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fc125-5b32-424d-bef0-a7dd8d2e79ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.599881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fc125-5b32-424d-bef0-a7dd8d2e79ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.602307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fc125-5b32-424d-bef0-a7dd8d2e79ed admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.650760 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 982/1939] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:02 2022] POST /identity/v3/projects => generated 389 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:02.654953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c47e4-a617-471b-ba37-233d741b1a56 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.664853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:02.717753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c47e4-a617-471b-ba37-233d741b1a56 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.725202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06caf3-cad7-49f7-916e-493997f60528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.726676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06caf3-cad7-49f7-916e-493997f60528 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.727405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06caf3-cad7-49f7-916e-493997f60528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.727405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06caf3-cad7-49f7-916e-493997f60528 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.727935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06caf3-cad7-49f7-916e-493997f60528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.729911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06caf3-cad7-49f7-916e-493997f60528 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.733380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 958/1940] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:02 2022] POST /identity/v3/projects => generated 365 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:02.737668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fc125-5b32-424d-bef0-a7dd8d2e79ed admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.820848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fc125-5b32-424d-bef0-a7dd8d2e79ed admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.825674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dedc83-e7bf-4538-af29-53e1fd4c5a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.826361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dedc83-e7bf-4538-af29-53e1fd4c5a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.826646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dedc83-e7bf-4538-af29-53e1fd4c5a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.826921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dedc83-e7bf-4538-af29-53e1fd4c5a15 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.827619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dedc83-e7bf-4538-af29-53e1fd4c5a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.829568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dedc83-e7bf-4538-af29-53e1fd4c5a15 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.850728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 983/1941] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:02 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:02.859335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06caf3-cad7-49f7-916e-493997f60528 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.866468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06caf3-cad7-49f7-916e-493997f60528 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.871573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0964a5-60be-4c40-b859-48b6123d21f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.872240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0964a5-60be-4c40-b859-48b6123d21f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.872523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0964a5-60be-4c40-b859-48b6123d21f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.872801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0964a5-60be-4c40-b859-48b6123d21f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.873558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0964a5-60be-4c40-b859-48b6123d21f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.875630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0964a5-60be-4c40-b859-48b6123d21f9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.890012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 984/1942] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:02.893994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0964a5-60be-4c40-b859-48b6123d21f9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.903870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0964a5-60be-4c40-b859-48b6123d21f9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.915704 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bffbc-962f-418c-aa1b-0d59aecaff3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.916506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bffbc-962f-418c-aa1b-0d59aecaff3e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.923965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bffbc-962f-418c-aa1b-0d59aecaff3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.924297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bffbc-962f-418c-aa1b-0d59aecaff3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.926147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bffbc-962f-418c-aa1b-0d59aecaff3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.927808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bffbc-962f-418c-aa1b-0d59aecaff3e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.939289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 985/1943] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:02.951487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bffbc-962f-418c-aa1b-0d59aecaff3e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:02.953327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bffbc-962f-418c-aa1b-0d59aecaff3e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:02.958013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc47a8-ba1a-4297-8d00-77fce38a1853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:02.958881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc47a8-ba1a-4297-8d00-77fce38a1853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:02.959178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc47a8-ba1a-4297-8d00-77fce38a1853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:02.959436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc47a8-ba1a-4297-8d00-77fce38a1853 admin admin] PATH_INFO: `/v3/projects/ab4bf6f655444c1daae3b0a648e44bd4/users/c9cb1f029bbe4c1884d0d232e73888e0/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:02.974399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc47a8-ba1a-4297-8d00-77fce38a1853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab4bf6f655444c1daae3b0a648e44bd4, user_id=c9cb1f029bbe4c1884d0d232e73888e0, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:02.976952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc47a8-ba1a-4297-8d00-77fce38a1853 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:02.984264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 959/1944] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:02 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:02.989016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dedc83-e7bf-4538-af29-53e1fd4c5a15 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.000071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dedc83-e7bf-4538-af29-53e1fd4c5a15 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.008152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.008152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.008152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.008152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.008486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.009914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.023995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 960/1945] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:03.023995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.023995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:03.041340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5da9-c2de-4480-b976-7080b516ef9b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.046273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.047656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.047941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.048218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] PATH_INFO: `/v3/projects/8923fbaf1c8a48d9a41cf472a492c56d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.049441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8923fbaf1c8a48d9a41cf472a492c56d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.052073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.054642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:03.089844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 986/1946] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:02 2022] PUT /identity/v3/projects/ab4bf6f655444c1daae3b0a648e44bd4/users/c9cb1f029bbe4c1884d0d232e73888e0/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:03.090670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc47a8-ba1a-4297-8d00-77fce38a1853 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.163862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc47a8-ba1a-4297-8d00-77fce38a1853 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.168415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55ae12-e66e-4384-ba97-e44501386ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.169345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55ae12-e66e-4384-ba97-e44501386ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.173483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55ae12-e66e-4384-ba97-e44501386ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.173483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55ae12-e66e-4384-ba97-e44501386ed8 admin admin] PATH_INFO: `/v3/projects/9e5ebb1d486a4baf9a33d0930df1f45d/users/8d0c4d468a5b4040b8d65ac0c8ae947e/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.182681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8923fbaf1c8a48d9a41cf472a492c56d', 'request_id': 'req-297655ce-95b6-4306-a984-c09dfa72f672'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:03.186110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55ae12-e66e-4384-ba97-e44501386ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5ebb1d486a4baf9a33d0930df1f45d, user_id=8d0c4d468a5b4040b8d65ac0c8ae947e, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.188765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55ae12-e66e-4384-ba97-e44501386ed8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.192348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8923fbaf1c8a48d9a41cf472a492c56d', 'request_id': 'req-297655ce-95b6-4306-a984-c09dfa72f672'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:03.198043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8923fbaf1c8a48d9a41cf472a492c56d', 'request_id': 'req-297655ce-95b6-4306-a984-c09dfa72f672'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:03.280022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 987/1947] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:03 2022] PUT /identity/v3/projects/9e5ebb1d486a4baf9a33d0930df1f45d/users/8d0c4d468a5b4040b8d65ac0c8ae947e/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:03.281841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe73cc-636a-413d-a4db-b6e74d210c43 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.281950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] The token cache is being invalidate because project 8923fbaf1c8a48d9a41cf472a492c56d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:03.282130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe73cc-636a-413d-a4db-b6e74d210c43 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.282266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:03.284962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe73cc-636a-413d-a4db-b6e74d210c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.285062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:03.286117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 961/1948] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:03 2022] DELETE /identity/v3/projects/8923fbaf1c8a48d9a41cf472a492c56d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:03.287783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.291356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:03.317295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0dfe73cc-636a-413d-a4db-b6e74d210c43 None None] MFA Rules not processed for user `c9cb1f029bbe4c1884d0d232e73888e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:03.318640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0dfe73cc-636a-413d-a4db-b6e74d210c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:03.344348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:03.356750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297655ce-95b6-4306-a984-c09dfa72f672 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.390160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 988/1949] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:03.394087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738a0fc-22e9-4572-8a9a-a8d81a1b681d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.394087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738a0fc-22e9-4572-8a9a-a8d81a1b681d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.394087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738a0fc-22e9-4572-8a9a-a8d81a1b681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.410332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a738a0fc-22e9-4572-8a9a-a8d81a1b681d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:03.412604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a738a0fc-22e9-4572-8a9a-a8d81a1b681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:03.422863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.422863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.422863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.422863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.423571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.425789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.435802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 962/1950] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:03.436584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.440454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:03.452820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171154b0-8efc-4490-bfe9-5eec2e5a92cb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.458831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0321f-c65c-4316-8cf9-61e70986bd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.459697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0321f-c65c-4316-8cf9-61e70986bd49 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.459981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0321f-c65c-4316-8cf9-61e70986bd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.460253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0321f-c65c-4316-8cf9-61e70986bd49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.460953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0321f-c65c-4316-8cf9-61e70986bd49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.462926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0321f-c65c-4316-8cf9-61e70986bd49 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.478881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 989/1951] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:03.479713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738a0fc-22e9-4572-8a9a-a8d81a1b681d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.489776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738a0fc-22e9-4572-8a9a-a8d81a1b681d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.494745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ce266-6350-4a26-aa3c-54ddecd716b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.495866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ce266-6350-4a26-aa3c-54ddecd716b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.496153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ce266-6350-4a26-aa3c-54ddecd716b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.496487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ce266-6350-4a26-aa3c-54ddecd716b2 admin admin] PATH_INFO: `/v3/projects/9e5ebb1d486a4baf9a33d0930df1f45d/users/8d0c4d468a5b4040b8d65ac0c8ae947e/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.498684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ce266-6350-4a26-aa3c-54ddecd716b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5ebb1d486a4baf9a33d0930df1f45d, user_id=8d0c4d468a5b4040b8d65ac0c8ae947e, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.501484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ce266-6350-4a26-aa3c-54ddecd716b2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.512256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 990/1952] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:03 2022] PUT /identity/v3/projects/9e5ebb1d486a4baf9a33d0930df1f45d/users/8d0c4d468a5b4040b8d65ac0c8ae947e/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:03.513520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9a40-bc1a-4890-9873-55c15f6b0a8d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.513813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9a40-bc1a-4890-9873-55c15f6b0a8d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.514098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f9a40-bc1a-4890-9873-55c15f6b0a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.532413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0e3f9a40-bc1a-4890-9873-55c15f6b0a8d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:03.533669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0e3f9a40-bc1a-4890-9873-55c15f6b0a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:03.585233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 963/1953] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:03 2022] POST /identity/v3/projects => generated 389 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:03.585436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4710-163f-4b0a-91b6-2a908c49d526 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.585657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4710-163f-4b0a-91b6-2a908c49d526 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.585932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4710-163f-4b0a-91b6-2a908c49d526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.601182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 991/1954] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:03.601989 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f9a40-bc1a-4890-9873-55c15f6b0a8d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.611404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f9a40-bc1a-4890-9873-55c15f6b0a8d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.613379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:03.616421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0dec4710-163f-4b0a-91b6-2a908c49d526 None None] MFA Rules not processed for user `8d0c4d468a5b4040b8d65ac0c8ae947e`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:03.617654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0dec4710-163f-4b0a-91b6-2a908c49d526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:03.662573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:03.672756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4363f-a61d-4c28-b046-b80091bf6889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.673540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4363f-a61d-4c28-b046-b80091bf6889 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.673771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4363f-a61d-4c28-b046-b80091bf6889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.674051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4363f-a61d-4c28-b046-b80091bf6889 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.674772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4363f-a61d-4c28-b046-b80091bf6889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.676710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4363f-a61d-4c28-b046-b80091bf6889 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.685401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 964/1955] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:03.686405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4710-163f-4b0a-91b6-2a908c49d526 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.696731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4710-163f-4b0a-91b6-2a908c49d526 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.701388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.702050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.702340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.702616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.703343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:03.703652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.705294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.717590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 965/1956] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:03.718388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.720008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:03.778812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71142365-0018-4b95-b156-f7d690f1d899 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.785344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 992/1957] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:03 2022] POST /identity/v3/users => generated 301 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:03.785499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8979e79-1d55-43b3-9a61-4f4e656e753e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.785623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4363f-a61d-4c28-b046-b80091bf6889 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.785721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8979e79-1d55-43b3-9a61-4f4e656e753e service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.785721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8979e79-1d55-43b3-9a61-4f4e656e753e service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.785721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8979e79-1d55-43b3-9a61-4f4e656e753e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.794692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4363f-a61d-4c28-b046-b80091bf6889 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.795232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8979e79-1d55-43b3-9a61-4f4e656e753e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.797001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8979e79-1d55-43b3-9a61-4f4e656e753e service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.802996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267b7bd-80e7-4b9c-9c15-edb42f5adc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.803958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267b7bd-80e7-4b9c-9c15-edb42f5adc27 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.806917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267b7bd-80e7-4b9c-9c15-edb42f5adc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.806917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267b7bd-80e7-4b9c-9c15-edb42f5adc27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.806917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267b7bd-80e7-4b9c-9c15-edb42f5adc27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.807161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267b7bd-80e7-4b9c-9c15-edb42f5adc27 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.808049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 966/1958] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:03.808870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8979e79-1d55-43b3-9a61-4f4e656e753e service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.819862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8979e79-1d55-43b3-9a61-4f4e656e753e service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.825431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c464b1-3314-456e-b538-1a54600f1211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.826237 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c464b1-3314-456e-b538-1a54600f1211 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.826521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c464b1-3314-456e-b538-1a54600f1211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.826813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c464b1-3314-456e-b538-1a54600f1211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.827650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c464b1-3314-456e-b538-1a54600f1211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.829292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c464b1-3314-456e-b538-1a54600f1211 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.840483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 967/1959] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:03.844653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c464b1-3314-456e-b538-1a54600f1211 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.855060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c464b1-3314-456e-b538-1a54600f1211 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:03.859883 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09218c6-ae61-47be-a7a4-942a231448b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:03.860825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09218c6-ae61-47be-a7a4-942a231448b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:03.861057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09218c6-ae61-47be-a7a4-942a231448b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:03.861367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09218c6-ae61-47be-a7a4-942a231448b6 admin admin] PATH_INFO: `/v3/projects/f44dea62c4ba44b492b6b0a61e73aa35/users/4a65caee4b22433dac02b085f9c49d76/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:03.877640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09218c6-ae61-47be-a7a4-942a231448b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44dea62c4ba44b492b6b0a61e73aa35, user_id=4a65caee4b22433dac02b085f9c49d76, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:03.880313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09218c6-ae61-47be-a7a4-942a231448b6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:03.898117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 993/1960] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:03 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:03.902126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267b7bd-80e7-4b9c-9c15-edb42f5adc27 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.960398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 968/1961] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:03 2022] PUT /identity/v3/projects/f44dea62c4ba44b492b6b0a61e73aa35/users/4a65caee4b22433dac02b085f9c49d76/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:03.964381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09218c6-ae61-47be-a7a4-942a231448b6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:03.979555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267b7bd-80e7-4b9c-9c15-edb42f5adc27 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.021683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:04.031741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09218c6-ae61-47be-a7a4-942a231448b6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.037255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c7161-2b24-448e-bb75-63636e86c13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.038046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c7161-2b24-448e-bb75-63636e86c13e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.038379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c7161-2b24-448e-bb75-63636e86c13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.038660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c7161-2b24-448e-bb75-63636e86c13e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.039420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c7161-2b24-448e-bb75-63636e86c13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.041147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c7161-2b24-448e-bb75-63636e86c13e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.048996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0e0e2-f755-4aa2-b194-150d44eca4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.049750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0e0e2-f755-4aa2-b194-150d44eca4e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.050037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0e0e2-f755-4aa2-b194-150d44eca4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.050315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0e0e2-f755-4aa2-b194-150d44eca4e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.051200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0e0e2-f755-4aa2-b194-150d44eca4e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.054894 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 969/1962] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:04.055041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0e0e2-f755-4aa2-b194-150d44eca4e7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.057565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c7161-2b24-448e-bb75-63636e86c13e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:04.069561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c7161-2b24-448e-bb75-63636e86c13e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.076764 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30390525-bc44-44f3-a98c-81dd297b3584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.077731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390525-bc44-44f3-a98c-81dd297b3584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.078583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390525-bc44-44f3-a98c-81dd297b3584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.079241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30390525-bc44-44f3-a98c-81dd297b3584 admin admin] PATH_INFO: `/v3/projects/f44dea62c4ba44b492b6b0a61e73aa35/users/4a65caee4b22433dac02b085f9c49d76/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.089217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30390525-bc44-44f3-a98c-81dd297b3584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44dea62c4ba44b492b6b0a61e73aa35, user_id=4a65caee4b22433dac02b085f9c49d76, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.089217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30390525-bc44-44f3-a98c-81dd297b3584 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.098881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 970/1963] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:04 2022] PUT /identity/v3/projects/f44dea62c4ba44b492b6b0a61e73aa35/users/4a65caee4b22433dac02b085f9c49d76/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:04.110949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33458a46-993f-4357-99df-4e97ad0a6e30 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.111273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33458a46-993f-4357-99df-4e97ad0a6e30 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.111565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33458a46-993f-4357-99df-4e97ad0a6e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.144612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-33458a46-993f-4357-99df-4e97ad0a6e30 None None] MFA Rules not processed for user `4a65caee4b22433dac02b085f9c49d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:04.146302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-33458a46-993f-4357-99df-4e97ad0a6e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:04.148118 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 994/1964] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:03 2022] POST /identity/v3/users => generated 284 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:04.152182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0e0e2-f755-4aa2-b194-150d44eca4e7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:04.163002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0e0e2-f755-4aa2-b194-150d44eca4e7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.217463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:04.227962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5d43-f1b7-4ccb-a358-2e83280969cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.228907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5d43-f1b7-4ccb-a358-2e83280969cf admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.229226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5d43-f1b7-4ccb-a358-2e83280969cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.229612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea5d43-f1b7-4ccb-a358-2e83280969cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.230687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea5d43-f1b7-4ccb-a358-2e83280969cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.235073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea5d43-f1b7-4ccb-a358-2e83280969cf admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.237357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 971/1965] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:04.246990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 995/1966] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:04.255971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33458a46-993f-4357-99df-4e97ad0a6e30 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:04.262706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5d43-f1b7-4ccb-a358-2e83280969cf admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:04.270557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33458a46-993f-4357-99df-4e97ad0a6e30 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.276374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4725989-9677-488e-9c3f-8548a201f6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.277618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4725989-9677-488e-9c3f-8548a201f6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.277927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4725989-9677-488e-9c3f-8548a201f6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.278206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4725989-9677-488e-9c3f-8548a201f6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.278933 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4725989-9677-488e-9c3f-8548a201f6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.280951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4725989-9677-488e-9c3f-8548a201f6c2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.295408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 972/1967] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:04.299483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4725989-9677-488e-9c3f-8548a201f6c2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:04.310267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4725989-9677-488e-9c3f-8548a201f6c2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.315124 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943d1d1-c782-45a1-b5fc-c7239fbb5b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.316050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d1d1-c782-45a1-b5fc-c7239fbb5b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.316330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d1d1-c782-45a1-b5fc-c7239fbb5b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.316600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d1d1-c782-45a1-b5fc-c7239fbb5b90 admin admin] PATH_INFO: `/v3/projects/e50dec77ce95482684238cccd67838d8/users/e6e9cfec6bec43b6a17aba51cc0f991d/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.325525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea5d43-f1b7-4ccb-a358-2e83280969cf admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.328565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94771734-dbfd-4a5e-9064-11a47770fba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.332290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771734-dbfd-4a5e-9064-11a47770fba2 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.332290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771734-dbfd-4a5e-9064-11a47770fba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.332290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94771734-dbfd-4a5e-9064-11a47770fba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.334056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943d1d1-c782-45a1-b5fc-c7239fbb5b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50dec77ce95482684238cccd67838d8, user_id=e6e9cfec6bec43b6a17aba51cc0f991d, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.336808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943d1d1-c782-45a1-b5fc-c7239fbb5b90 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.340993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94771734-dbfd-4a5e-9064-11a47770fba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.342508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94771734-dbfd-4a5e-9064-11a47770fba2 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.353037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 996/1968] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:04.438607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 973/1969] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:04 2022] PUT /identity/v3/projects/e50dec77ce95482684238cccd67838d8/users/e6e9cfec6bec43b6a17aba51cc0f991d/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:04.449445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890dce8-207b-42a9-be9d-8cac59d195d2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.449742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890dce8-207b-42a9-be9d-8cac59d195d2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.450021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890dce8-207b-42a9-be9d-8cac59d195d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.488955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7890dce8-207b-42a9-be9d-8cac59d195d2 None None] MFA Rules not processed for user `e6e9cfec6bec43b6a17aba51cc0f991d`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:04.488955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7890dce8-207b-42a9-be9d-8cac59d195d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:04.521997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943d1d1-c782-45a1-b5fc-c7239fbb5b90 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:04.568489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 997/1970] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:04.590283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890dce8-207b-42a9-be9d-8cac59d195d2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:04.593882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7890dce8-207b-42a9-be9d-8cac59d195d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:04.594766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943d1d1-c782-45a1-b5fc-c7239fbb5b90 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.600160 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.601046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.601370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.601648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] PATH_INFO: `/v3/users/c9cb1f029bbe4c1884d0d232e73888e0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.602874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9cb1f029bbe4c1884d0d232e73888e0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.604828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.660199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890dce8-207b-42a9-be9d-8cac59d195d2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.663865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33fcd3-f195-484d-af5c-f39ce5fb141b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47589bf9e7c4acba200979e3a91a6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.664512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33fcd3-f195-484d-af5c-f39ce5fb141b service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.664797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33fcd3-f195-484d-af5c-f39ce5fb141b service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.665079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33fcd3-f195-484d-af5c-f39ce5fb141b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.674494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33fcd3-f195-484d-af5c-f39ce5fb141b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.675965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33fcd3-f195-484d-af5c-f39ce5fb141b service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.686120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 998/1971] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:04.703026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:04.791598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9cb1f029bbe4c1884d0d232e73888e0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:04.804646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9cb1f029bbe4c1884d0d232e73888e0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:04.805455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9cb1f029bbe4c1884d0d232e73888e0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:04.806175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9cb1f029bbe4c1884d0d232e73888e0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:04.838690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33fcd3-f195-484d-af5c-f39ce5fb141b service glance] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:04.849749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33fcd3-f195-484d-af5c-f39ce5fb141b service glance] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:04.855275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:04.856172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.856453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.856723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] PATH_INFO: `/v3/users/82a0db16d583443cb30058d3ef774103` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.858071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82a0db16d583443cb30058d3ef774103)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:04.860057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:04.885834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:04.896860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-213feebf-c808-4794-9122-07bdabe41dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9cb1f029bbe4c1884d0d232e73888e0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:04.913932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 974/1972] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:04 2022] DELETE /identity/v3/users/c9cb1f029bbe4c1884d0d232e73888e0 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:04.917784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a5e9e-82ba-4a22-abe8-b3257ee31f3d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:04.918078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a5e9e-82ba-4a22-abe8-b3257ee31f3d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:04.918370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a5e9e-82ba-4a22-abe8-b3257ee31f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:04.937685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-273a5e9e-82ba-4a22-abe8-b3257ee31f3d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:04.938823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-273a5e9e-82ba-4a22-abe8-b3257ee31f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:05.011274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 975/1973] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:05.021580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a5e9e-82ba-4a22-abe8-b3257ee31f3d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:05.022944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-273a5e9e-82ba-4a22-abe8-b3257ee31f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:05.046133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82a0db16d583443cb30058d3ef774103'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:05.053355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82a0db16d583443cb30058d3ef774103'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:05.065620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82a0db16d583443cb30058d3ef774103'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:05.066362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82a0db16d583443cb30058d3ef774103'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:05.084197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a5e9e-82ba-4a22-abe8-b3257ee31f3d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:05.088141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cad63-5776-42f2-a1a6-ec10fbf9df8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:05.089040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cad63-5776-42f2-a1a6-ec10fbf9df8f service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:05.089328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cad63-5776-42f2-a1a6-ec10fbf9df8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:05.089608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cad63-5776-42f2-a1a6-ec10fbf9df8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:05.100191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cad63-5776-42f2-a1a6-ec10fbf9df8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:05.102072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cad63-5776-42f2-a1a6-ec10fbf9df8f service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:05.111696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 976/1974] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:05.130311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82a0db16d583443cb30058d3ef774103'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:05.137096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 999/1975] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:04 2022] DELETE /identity/v3/users/82a0db16d583443cb30058d3ef774103 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:05.141544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513f6a2-ddae-4835-b213-5ec8ddb43bcf None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:05.141842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513f6a2-ddae-4835-b213-5ec8ddb43bcf None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:05.142127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513f6a2-ddae-4835-b213-5ec8ddb43bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:05.167603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3513f6a2-ddae-4835-b213-5ec8ddb43bcf None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:05.168882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3513f6a2-ddae-4835-b213-5ec8ddb43bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:05.222765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:05.240291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 977/1976] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:05.250129 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:05.251569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:05.313071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a0fff-f137-4185-bd6e-fa2cb7bff539 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:05.317881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:05.318853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:05.319100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:05.322207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:05.330372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:05.332384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:05.342812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1000/1977] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:05.761871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513f6a2-ddae-4835-b213-5ec8ddb43bcf None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:05.768623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3513f6a2-ddae-4835-b213-5ec8ddb43bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:05.785684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513f6a2-ddae-4835-b213-5ec8ddb43bcf None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:05.790657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:05.791526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:05.791805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:05.792077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] PATH_INFO: `/v3/projects/ab4bf6f655444c1daae3b0a648e44bd4` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:05.793309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab4bf6f655444c1daae3b0a648e44bd4)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:05.795223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:05.886742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:05.888200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:05.904542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc670cb9-3171-49e3-bd44-46e7a9032efd service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:05.911469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:05.912599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:05.912978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:05.913440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] PATH_INFO: `/v3/projects/2849d47d98844282a331ba9f5abc0ed9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:05.914827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2849d47d98844282a331ba9f5abc0ed9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:05.916842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:05.966056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab4bf6f655444c1daae3b0a648e44bd4', 'request_id': 'req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:05.972594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab4bf6f655444c1daae3b0a648e44bd4', 'request_id': 'req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:05.978219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab4bf6f655444c1daae3b0a648e44bd4', 'request_id': 'req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.058869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2849d47d98844282a331ba9f5abc0ed9', 'request_id': 'req-815ea6bf-2b83-423c-954e-1ae7b14c017a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.070231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2849d47d98844282a331ba9f5abc0ed9', 'request_id': 'req-815ea6bf-2b83-423c-954e-1ae7b14c017a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.075791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] The token cache is being invalidate because project ab4bf6f655444c1daae3b0a648e44bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:06.076087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.076373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2849d47d98844282a331ba9f5abc0ed9', 'request_id': 'req-815ea6bf-2b83-423c-954e-1ae7b14c017a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.076727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.078351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 978/1978] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:05 2022] DELETE /identity/v3/projects/ab4bf6f655444c1daae3b0a648e44bd4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:06.082531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:06.083907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:06.151576 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3eddf-fbc4-434f-b9d1-c8bbc928a1dd admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:06.209481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:06.213177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:06.213177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:06.213177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] PATH_INFO: `/v3/users/4a65caee4b22433dac02b085f9c49d76` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:06.213177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a65caee4b22433dac02b085f9c49d76)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:06.214047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:06.235784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:06.247567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] The token cache is being invalidate because project 2849d47d98844282a331ba9f5abc0ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:06.247567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.247567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.247567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1001/1979] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:05 2022] DELETE /identity/v3/projects/2849d47d98844282a331ba9f5abc0ed9 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:06.247567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:06.247949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:06.260587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ea6bf-2b83-423c-954e-1ae7b14c017a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:06.265423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:06.266332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:06.266622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:06.266905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] PATH_INFO: `/v3/users/8d0c4d468a5b4040b8d65ac0c8ae947e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:06.268133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0c4d468a5b4040b8d65ac0c8ae947e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:06.270110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:06.294098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:06.505593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a65caee4b22433dac02b085f9c49d76'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.529334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a65caee4b22433dac02b085f9c49d76'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.530126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a65caee4b22433dac02b085f9c49d76'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.530759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a65caee4b22433dac02b085f9c49d76'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.542624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0c4d468a5b4040b8d65ac0c8ae947e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.550447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0c4d468a5b4040b8d65ac0c8ae947e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.563343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0c4d468a5b4040b8d65ac0c8ae947e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.564095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0c4d468a5b4040b8d65ac0c8ae947e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.641484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a65caee4b22433dac02b085f9c49d76'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.664379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 979/1980] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:06 2022] DELETE /identity/v3/users/4a65caee4b22433dac02b085f9c49d76 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:06.695798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0c4d468a5b4040b8d65ac0c8ae947e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:06.708006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1002/1981] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:06 2022] DELETE /identity/v3/users/8d0c4d468a5b4040b8d65ac0c8ae947e => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:07.179929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:07.180289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:07.180755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:07.181439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:07.269456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9bbdb-7801-4985-b315-5f650e4c339f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:07.274385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:07.275236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.275510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.275777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] PATH_INFO: `/v3/projects/f44dea62c4ba44b492b6b0a61e73aa35` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.277002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44dea62c4ba44b492b6b0a61e73aa35)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:07.279051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:07.279163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d7930-064b-4d5d-a49e-58ec4f2d3084 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:07.284001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:07.284958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.285446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.285733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] PATH_INFO: `/v3/projects/9e5ebb1d486a4baf9a33d0930df1f45d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.286981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5ebb1d486a4baf9a33d0930df1f45d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:07.289899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:07.438635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44dea62c4ba44b492b6b0a61e73aa35', 'request_id': 'req-f27313fc-96d1-432c-894b-8f323206b57e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.439541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5ebb1d486a4baf9a33d0930df1f45d', 'request_id': 'req-591e212d-a438-4ac1-96cf-7754c30a0478'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.445575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44dea62c4ba44b492b6b0a61e73aa35', 'request_id': 'req-f27313fc-96d1-432c-894b-8f323206b57e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.450768 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5ebb1d486a4baf9a33d0930df1f45d', 'request_id': 'req-591e212d-a438-4ac1-96cf-7754c30a0478'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.452814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44dea62c4ba44b492b6b0a61e73aa35', 'request_id': 'req-f27313fc-96d1-432c-894b-8f323206b57e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.458717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5ebb1d486a4baf9a33d0930df1f45d', 'request_id': 'req-591e212d-a438-4ac1-96cf-7754c30a0478'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.557710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] The token cache is being invalidate because project f44dea62c4ba44b492b6b0a61e73aa35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:07.558082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.558987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f27313fc-96d1-432c-894b-8f323206b57e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.561113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 980/1982] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:07 2022] DELETE /identity/v3/projects/f44dea62c4ba44b492b6b0a61e73aa35 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:07.567479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df201c-1b14-4b83-a3e3-b0842703846c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.567898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df201c-1b14-4b83-a3e3-b0842703846c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.568307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df201c-1b14-4b83-a3e3-b0842703846c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.593399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-18df201c-1b14-4b83-a3e3-b0842703846c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:07.594600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-18df201c-1b14-4b83-a3e3-b0842703846c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:07.619954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:07.653871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] The token cache is being invalidate because project 9e5ebb1d486a4baf9a33d0930df1f45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:07.654161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.654781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-591e212d-a438-4ac1-96cf-7754c30a0478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:07.655161 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 981/1983] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:07.656786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1003/1984] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:07 2022] DELETE /identity/v3/projects/9e5ebb1d486a4baf9a33d0930df1f45d => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:07.662029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578afaf7-4947-495c-a9b4-f5ed60bac9b5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.662326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578afaf7-4947-495c-a9b4-f5ed60bac9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.662613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578afaf7-4947-495c-a9b4-f5ed60bac9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.663065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127563d-0661-4185-bab7-b55786efa12d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.663388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127563d-0661-4185-bab7-b55786efa12d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.663672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127563d-0661-4185-bab7-b55786efa12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.682123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-578afaf7-4947-495c-a9b4-f5ed60bac9b5 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:07.683348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-578afaf7-4947-495c-a9b4-f5ed60bac9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:07.685647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6127563d-0661-4185-bab7-b55786efa12d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:07.686999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6127563d-0661-4185-bab7-b55786efa12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:07.695600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1004/1985] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:07.697252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 982/1986] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:07.699622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578afaf7-4947-495c-a9b4-f5ed60bac9b5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:07.699958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b854d8-7403-4285-9636-b1e876f429ba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.700243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b854d8-7403-4285-9636-b1e876f429ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.700521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b854d8-7403-4285-9636-b1e876f429ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.709940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578afaf7-4947-495c-a9b4-f5ed60bac9b5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:07.722567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-91b854d8-7403-4285-9636-b1e876f429ba None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:07.724110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-91b854d8-7403-4285-9636-b1e876f429ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:07.728355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:07.729039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.729375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.729659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.744757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:07.744757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:07.744757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:07.749260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1005/1987] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:07.752247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b854d8-7403-4285-9636-b1e876f429ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:07.756963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 983/1988] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:37:07.757958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:07.766768 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b854d8-7403-4285-9636-b1e876f429ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:07.769325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322231f-8bbc-491c-8ba0-e8cb23542932 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:07.774800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30ad08-315a-49c8-8dc2-8ed2d48faedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:07.775438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:07.775564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30ad08-315a-49c8-8dc2-8ed2d48faedc admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.775832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30ad08-315a-49c8-8dc2-8ed2d48faedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.776036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30ad08-315a-49c8-8dc2-8ed2d48faedc admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.776136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.776342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.776619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.776729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30ad08-315a-49c8-8dc2-8ed2d48faedc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:07.777379 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:07.777700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:07.778774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30ad08-315a-49c8-8dc2-8ed2d48faedc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:07.779274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:07.793934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1006/1989] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:37:07.798256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:07.804062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:07.809491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838ac01-1d1c-4335-9e09-8b604b004825 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:07.814315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09a71-afff-49c6-af90-a1ef37fe91a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:07.815004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09a71-afff-49c6-af90-a1ef37fe91a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:07.815316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09a71-afff-49c6-af90-a1ef37fe91a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:07.815624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c09a71-afff-49c6-af90-a1ef37fe91a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:07.816310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c09a71-afff-49c6-af90-a1ef37fe91a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:07.818246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c09a71-afff-49c6-af90-a1ef37fe91a0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:07.920322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 984/1990] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:07 2022] POST /identity/v3/projects => generated 409 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:07.921304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30ad08-315a-49c8-8dc2-8ed2d48faedc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:07.942340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1007/1991] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:07 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:07.950714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09a71-afff-49c6-af90-a1ef37fe91a0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:07.985316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30ad08-315a-49c8-8dc2-8ed2d48faedc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.020338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c09a71-afff-49c6-af90-a1ef37fe91a0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.025093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e1e03-2e04-4ade-8d3d-979ff1e01cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.025792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1e03-2e04-4ade-8d3d-979ff1e01cac admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.026077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1e03-2e04-4ade-8d3d-979ff1e01cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.026358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e1e03-2e04-4ade-8d3d-979ff1e01cac admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.027044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e1e03-2e04-4ade-8d3d-979ff1e01cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.029064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e1e03-2e04-4ade-8d3d-979ff1e01cac admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.046230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb79d2d-708c-4e00-8206-b6cc44cc456f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.046906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79d2d-708c-4e00-8206-b6cc44cc456f admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.047189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79d2d-708c-4e00-8206-b6cc44cc456f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.047462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79d2d-708c-4e00-8206-b6cc44cc456f admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.048196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb79d2d-708c-4e00-8206-b6cc44cc456f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.050844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb79d2d-708c-4e00-8206-b6cc44cc456f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.129319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1008/1992] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:07 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:08.133382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e1e03-2e04-4ade-8d3d-979ff1e01cac admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.142791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e1e03-2e04-4ade-8d3d-979ff1e01cac admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.147845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34321d-3c5b-4cf9-936e-ab59d95497f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.148468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34321d-3c5b-4cf9-936e-ab59d95497f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.148750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34321d-3c5b-4cf9-936e-ab59d95497f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.149027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34321d-3c5b-4cf9-936e-ab59d95497f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.150014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34321d-3c5b-4cf9-936e-ab59d95497f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.151748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34321d-3c5b-4cf9-936e-ab59d95497f2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.157515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 985/1993] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:07 2022] POST /identity/v3/users => generated 305 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:08.163234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1009/1994] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:08.166257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb79d2d-708c-4e00-8206-b6cc44cc456f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.166845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34321d-3c5b-4cf9-936e-ab59d95497f2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.177536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb79d2d-708c-4e00-8206-b6cc44cc456f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.177971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34321d-3c5b-4cf9-936e-ab59d95497f2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.182380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fad0b4-4a1c-48de-bab7-1c0ca79f9ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.182899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77033-96bd-438f-9732-f6e292aab998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.183051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fad0b4-4a1c-48de-bab7-1c0ca79f9ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.183296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77033-96bd-438f-9732-f6e292aab998 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.183388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fad0b4-4a1c-48de-bab7-1c0ca79f9ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.183598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77033-96bd-438f-9732-f6e292aab998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.183700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fad0b4-4a1c-48de-bab7-1c0ca79f9ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.183865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77033-96bd-438f-9732-f6e292aab998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.184303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fad0b4-4a1c-48de-bab7-1c0ca79f9ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.184583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db77033-96bd-438f-9732-f6e292aab998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.186034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fad0b4-4a1c-48de-bab7-1c0ca79f9ad0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.189395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db77033-96bd-438f-9732-f6e292aab998 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.199923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 986/1995] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:08.200099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1010/1996] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:08.200928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fad0b4-4a1c-48de-bab7-1c0ca79f9ad0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.201698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77033-96bd-438f-9732-f6e292aab998 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.212817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77033-96bd-438f-9732-f6e292aab998 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.214625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fad0b4-4a1c-48de-bab7-1c0ca79f9ad0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.218063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a120a-6162-4e12-a3f2-9846fa79c75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.218964 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a120a-6162-4e12-a3f2-9846fa79c75a admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.219151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a99e36-e954-4a1e-9c17-4fd77d2c527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.222869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a120a-6162-4e12-a3f2-9846fa79c75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.222869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a120a-6162-4e12-a3f2-9846fa79c75a admin admin] PATH_INFO: `/v3/projects/a932c0c29ac746918db9d77537693066/users/f65c64f45b514865a1b46317c6519603/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.223060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99e36-e954-4a1e-9c17-4fd77d2c527e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.223060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99e36-e954-4a1e-9c17-4fd77d2c527e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.223060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a99e36-e954-4a1e-9c17-4fd77d2c527e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.223060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a99e36-e954-4a1e-9c17-4fd77d2c527e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.223701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a99e36-e954-4a1e-9c17-4fd77d2c527e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.238233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 987/1997] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:08.238415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a120a-6162-4e12-a3f2-9846fa79c75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a932c0c29ac746918db9d77537693066, user_id=f65c64f45b514865a1b46317c6519603, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.240835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a120a-6162-4e12-a3f2-9846fa79c75a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.241848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a99e36-e954-4a1e-9c17-4fd77d2c527e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.251475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a99e36-e954-4a1e-9c17-4fd77d2c527e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.259519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46815fe-8e6c-4b9b-bddf-9574e7accaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.260387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46815fe-8e6c-4b9b-bddf-9574e7accaa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.260669 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46815fe-8e6c-4b9b-bddf-9574e7accaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.260942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46815fe-8e6c-4b9b-bddf-9574e7accaa2 admin admin] PATH_INFO: `/v3/projects/7078de2623d2496991b394867371614b/users/98b525adfe144b26ae9608eb838045bc/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.277908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46815fe-8e6c-4b9b-bddf-9574e7accaa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7078de2623d2496991b394867371614b, user_id=98b525adfe144b26ae9608eb838045bc, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.280581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46815fe-8e6c-4b9b-bddf-9574e7accaa2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.324390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1011/1998] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:08 2022] PUT /identity/v3/projects/a932c0c29ac746918db9d77537693066/users/f65c64f45b514865a1b46317c6519603/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:08.336446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c469d-8537-4b88-a906-5120ab3e2d28 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.336745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c469d-8537-4b88-a906-5120ab3e2d28 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.337024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c469d-8537-4b88-a906-5120ab3e2d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.360977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 988/1999] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:08 2022] PUT /identity/v3/projects/7078de2623d2496991b394867371614b/users/98b525adfe144b26ae9608eb838045bc/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:08.365528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-da4c469d-8537-4b88-a906-5120ab3e2d28 None None] MFA Rules not processed for user `f65c64f45b514865a1b46317c6519603`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:08.366745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-da4c469d-8537-4b88-a906-5120ab3e2d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:08.372229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454e91-66a1-4022-a188-dd637e787636 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.372521 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454e91-66a1-4022-a188-dd637e787636 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.372805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d454e91-66a1-4022-a188-dd637e787636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.399895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0d454e91-66a1-4022-a188-dd637e787636 None None] MFA Rules not processed for user `98b525adfe144b26ae9608eb838045bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:08.401124 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0d454e91-66a1-4022-a188-dd637e787636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:08.435353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1012/2000] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:08.436512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c469d-8537-4b88-a906-5120ab3e2d28 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.469500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 989/2001] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:08.473856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454e91-66a1-4022-a188-dd637e787636 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.503939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c469d-8537-4b88-a906-5120ab3e2d28 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.504151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:08.514382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7935a-5d3d-4bb9-b0a7-311d958f38a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.515053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7935a-5d3d-4bb9-b0a7-311d958f38a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.515342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7935a-5d3d-4bb9-b0a7-311d958f38a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.516027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7935a-5d3d-4bb9-b0a7-311d958f38a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.516876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7935a-5d3d-4bb9-b0a7-311d958f38a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.519291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7935a-5d3d-4bb9-b0a7-311d958f38a8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.540068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d454e91-66a1-4022-a188-dd637e787636 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.544853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f90a63-a2d7-42cb-b362-bcf388c42985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.545550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f90a63-a2d7-42cb-b362-bcf388c42985 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.546023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f90a63-a2d7-42cb-b362-bcf388c42985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.546491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f90a63-a2d7-42cb-b362-bcf388c42985 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.547279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f90a63-a2d7-42cb-b362-bcf388c42985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.549561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f90a63-a2d7-42cb-b362-bcf388c42985 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.564438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:08.599147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1013/2002] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:08 2022] POST /identity/v3/projects => generated 363 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:08.603307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7935a-5d3d-4bb9-b0a7-311d958f38a8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.639659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 990/2003] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:08 2022] POST /identity/v3/projects => generated 409 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:08.644642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f90a63-a2d7-42cb-b362-bcf388c42985 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.678273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7935a-5d3d-4bb9-b0a7-311d958f38a8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.712752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f90a63-a2d7-42cb-b362-bcf388c42985 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.717968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789cb15-39df-4304-9a0f-d0e9b0d11699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.718646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789cb15-39df-4304-9a0f-d0e9b0d11699 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.718928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789cb15-39df-4304-9a0f-d0e9b0d11699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.719205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789cb15-39df-4304-9a0f-d0e9b0d11699 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.719968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789cb15-39df-4304-9a0f-d0e9b0d11699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.721994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789cb15-39df-4304-9a0f-d0e9b0d11699 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.735852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:08.736548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f95435-42ae-4a6c-9dbd-c7b9482322e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.737540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95435-42ae-4a6c-9dbd-c7b9482322e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.737823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95435-42ae-4a6c-9dbd-c7b9482322e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.738112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f95435-42ae-4a6c-9dbd-c7b9482322e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.738776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f95435-42ae-4a6c-9dbd-c7b9482322e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.740670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f95435-42ae-4a6c-9dbd-c7b9482322e9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.887770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 991/2004] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:08 2022] POST /identity/v3/users => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:08.891556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789cb15-39df-4304-9a0f-d0e9b0d11699 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.901409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1014/2005] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:08 2022] POST /identity/v3/users => generated 288 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:08.901602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789cb15-39df-4304-9a0f-d0e9b0d11699 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.913604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7026a3-aaa7-411d-b715-708480333875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.913604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7026a3-aaa7-411d-b715-708480333875 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.913604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7026a3-aaa7-411d-b715-708480333875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.913604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7026a3-aaa7-411d-b715-708480333875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.913979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7026a3-aaa7-411d-b715-708480333875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.913979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7026a3-aaa7-411d-b715-708480333875 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.915964 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f95435-42ae-4a6c-9dbd-c7b9482322e9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.920231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 992/2006] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:08.924481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7026a3-aaa7-411d-b715-708480333875 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.926622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f95435-42ae-4a6c-9dbd-c7b9482322e9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.932766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cad12-df30-42aa-bc77-1db5ae0b419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.933495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cad12-df30-42aa-bc77-1db5ae0b419b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.933781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cad12-df30-42aa-bc77-1db5ae0b419b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.934124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cad12-df30-42aa-bc77-1db5ae0b419b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.935078 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4cad12-df30-42aa-bc77-1db5ae0b419b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.935617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7026a3-aaa7-411d-b715-708480333875 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.937436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4cad12-df30-42aa-bc77-1db5ae0b419b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.949866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc31e4-6a6c-4eb4-b3bc-dc788bf35c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.955072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc31e4-6a6c-4eb4-b3bc-dc788bf35c3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.955362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc31e4-6a6c-4eb4-b3bc-dc788bf35c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.955640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc31e4-6a6c-4eb4-b3bc-dc788bf35c3b admin admin] PATH_INFO: `/v3/projects/cff5d8a1684949ea894fef882beffe5d/users/6f93be4a54ec44f8836d541adfbe3a65/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:08.960405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1015/2007] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:08.961505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cad12-df30-42aa-bc77-1db5ae0b419b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:08.976240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc31e4-6a6c-4eb4-b3bc-dc788bf35c3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff5d8a1684949ea894fef882beffe5d, user_id=6f93be4a54ec44f8836d541adfbe3a65, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:08.977314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4cad12-df30-42aa-bc77-1db5ae0b419b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:08.979031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc31e4-6a6c-4eb4-b3bc-dc788bf35c3b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:08.982345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68192f-2127-417c-bc4f-98c4eadf4ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:08.983040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68192f-2127-417c-bc4f-98c4eadf4ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:08.983371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68192f-2127-417c-bc4f-98c4eadf4ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:08.983650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68192f-2127-417c-bc4f-98c4eadf4ec3 admin admin] PATH_INFO: `/v3/projects/85945bd0af2447968d4c56eb14229f4e/users/002873ceedf346dea8c4a166022f946c/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.003628 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68192f-2127-417c-bc4f-98c4eadf4ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85945bd0af2447968d4c56eb14229f4e, user_id=002873ceedf346dea8c4a166022f946c, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:09.006364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68192f-2127-417c-bc4f-98c4eadf4ec3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:09.052253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 993/2008] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:08 2022] PUT /identity/v3/projects/cff5d8a1684949ea894fef882beffe5d/users/6f93be4a54ec44f8836d541adfbe3a65/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:09.056329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc31e4-6a6c-4eb4-b3bc-dc788bf35c3b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:09.074410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:09.082635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1016/2009] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:08 2022] PUT /identity/v3/projects/85945bd0af2447968d4c56eb14229f4e/users/002873ceedf346dea8c4a166022f946c/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:09.083973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68192f-2127-417c-bc4f-98c4eadf4ec3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:09.137579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc31e4-6a6c-4eb4-b3bc-dc788bf35c3b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:09.147207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68192f-2127-417c-bc4f-98c4eadf4ec3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:09.151905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c93678-6d0b-402b-b4d0-717982fd7be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:09.152570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93678-6d0b-402b-b4d0-717982fd7be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.152856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93678-6d0b-402b-b4d0-717982fd7be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.153278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93678-6d0b-402b-b4d0-717982fd7be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.154063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c93678-6d0b-402b-b4d0-717982fd7be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:09.155851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c93678-6d0b-402b-b4d0-717982fd7be0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:09.167297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1017/2010] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:09.171162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c93678-6d0b-402b-b4d0-717982fd7be0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:09.181436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c93678-6d0b-402b-b4d0-717982fd7be0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:09.186110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19409f-9775-4bb1-8877-5ed6a13f56c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:09.186974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19409f-9775-4bb1-8877-5ed6a13f56c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.187265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19409f-9775-4bb1-8877-5ed6a13f56c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.187543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19409f-9775-4bb1-8877-5ed6a13f56c5 admin admin] PATH_INFO: `/v3/projects/85945bd0af2447968d4c56eb14229f4e/users/002873ceedf346dea8c4a166022f946c/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.189538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19409f-9775-4bb1-8877-5ed6a13f56c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85945bd0af2447968d4c56eb14229f4e, user_id=002873ceedf346dea8c4a166022f946c, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:09.192311 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19409f-9775-4bb1-8877-5ed6a13f56c5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:09.193380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2413ad5-31a5-4d7d-9b8d-0576adca26ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:09.194048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2413ad5-31a5-4d7d-9b8d-0576adca26ff admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.194330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2413ad5-31a5-4d7d-9b8d-0576adca26ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.194607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2413ad5-31a5-4d7d-9b8d-0576adca26ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.195323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2413ad5-31a5-4d7d-9b8d-0576adca26ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:09.196992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2413ad5-31a5-4d7d-9b8d-0576adca26ff admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:09.203087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1018/2011] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:09 2022] PUT /identity/v3/projects/85945bd0af2447968d4c56eb14229f4e/users/002873ceedf346dea8c4a166022f946c/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:09.207246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 994/2012] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:09.214795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43a33-621c-41df-95c4-c1b41624fdf0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.214975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2413ad5-31a5-4d7d-9b8d-0576adca26ff admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:09.215092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43a33-621c-41df-95c4-c1b41624fdf0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.215357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf43a33-621c-41df-95c4-c1b41624fdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.225541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2413ad5-31a5-4d7d-9b8d-0576adca26ff admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:09.252996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5cf43a33-621c-41df-95c4-c1b41624fdf0 None None] MFA Rules not processed for user `002873ceedf346dea8c4a166022f946c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:09.252996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5cf43a33-621c-41df-95c4-c1b41624fdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:09.292906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817ed24-0feb-487a-a8ec-ad568ba73215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:09.293858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817ed24-0feb-487a-a8ec-ad568ba73215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.294138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817ed24-0feb-487a-a8ec-ad568ba73215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.294416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817ed24-0feb-487a-a8ec-ad568ba73215 admin admin] PATH_INFO: `/v3/projects/cff5d8a1684949ea894fef882beffe5d/users/6f93be4a54ec44f8836d541adfbe3a65/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.296389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817ed24-0feb-487a-a8ec-ad568ba73215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff5d8a1684949ea894fef882beffe5d, user_id=6f93be4a54ec44f8836d541adfbe3a65, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:09.299082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817ed24-0feb-487a-a8ec-ad568ba73215 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:09.303717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:09.312083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 995/2013] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:09 2022] PUT /identity/v3/projects/cff5d8a1684949ea894fef882beffe5d/users/6f93be4a54ec44f8836d541adfbe3a65/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:09.321987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b87bb-dd2c-48c0-b7a9-29150b7cf757 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.322277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b87bb-dd2c-48c0-b7a9-29150b7cf757 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.322657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b87bb-dd2c-48c0-b7a9-29150b7cf757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.332912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1019/2014] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:09.337919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf43a33-621c-41df-95c4-c1b41624fdf0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:09.339690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5cf43a33-621c-41df-95c4-c1b41624fdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:09.350309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8a0b87bb-dd2c-48c0-b7a9-29150b7cf757 None None] MFA Rules not processed for user `6f93be4a54ec44f8836d541adfbe3a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:09.351602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8a0b87bb-dd2c-48c0-b7a9-29150b7cf757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:09.396866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf43a33-621c-41df-95c4-c1b41624fdf0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:09.419746 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 996/2015] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:09 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:09.431375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b87bb-dd2c-48c0-b7a9-29150b7cf757 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:09.432928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8a0b87bb-dd2c-48c0-b7a9-29150b7cf757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:09.448194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dcece-64dc-42f5-ba58-bb2cafc5126f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:09.448938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dcece-64dc-42f5-ba58-bb2cafc5126f service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.449321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dcece-64dc-42f5-ba58-bb2cafc5126f service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.449675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dcece-64dc-42f5-ba58-bb2cafc5126f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.496914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b87bb-dd2c-48c0-b7a9-29150b7cf757 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:09.500686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c0083-5741-4c13-ac6c-1c314a7975ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:09.501361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0083-5741-4c13-ac6c-1c314a7975ac service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.501648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0083-5741-4c13-ac6c-1c314a7975ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.501927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c0083-5741-4c13-ac6c-1c314a7975ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.511233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c0083-5741-4c13-ac6c-1c314a7975ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:09.512638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c0083-5741-4c13-ac6c-1c314a7975ac service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:09.513081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5dcece-64dc-42f5-ba58-bb2cafc5126f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:09.514722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5dcece-64dc-42f5-ba58-bb2cafc5126f service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:09.523398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 997/2016] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:09 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:09.524910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1020/2017] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:09.553174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:09.788990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c0083-5741-4c13-ac6c-1c314a7975ac service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:09.797286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c0083-5741-4c13-ac6c-1c314a7975ac service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:09.800865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c081a-5bb3-466b-8b2f-e2c28a0a5edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:09.801527 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c081a-5bb3-466b-8b2f-e2c28a0a5edb service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.801810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c081a-5bb3-466b-8b2f-e2c28a0a5edb service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.802080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c081a-5bb3-466b-8b2f-e2c28a0a5edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:09.869411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c081a-5bb3-466b-8b2f-e2c28a0a5edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:09.870930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c081a-5bb3-466b-8b2f-e2c28a0a5edb service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:09.880708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 998/2018] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:09.930343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dcece-64dc-42f5-ba58-bb2cafc5126f service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:09.939139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dcece-64dc-42f5-ba58-bb2cafc5126f service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:09.943544 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c90ee-f063-4f3a-98f0-ad636c5ab278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:09.944173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c90ee-f063-4f3a-98f0-ad636c5ab278 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:09.944454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c90ee-f063-4f3a-98f0-ad636c5ab278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:09.944728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c90ee-f063-4f3a-98f0-ad636c5ab278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:10.004837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c90ee-f063-4f3a-98f0-ad636c5ab278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:10.006373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c90ee-f063-4f3a-98f0-ad636c5ab278 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:10.015386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1021/2019] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:09 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:19.927153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:20.011294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=67 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:23.385453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c081a-5bb3-466b-8b2f-e2c28a0a5edb service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:23.479196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c081a-5bb3-466b-8b2f-e2c28a0a5edb service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:23.483789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:23.484657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:23.484943 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:23.485250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] PATH_INFO: `/v3/users/98b525adfe144b26ae9608eb838045bc` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:23.486414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b525adfe144b26ae9608eb838045bc)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:23.489088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:23.680793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b525adfe144b26ae9608eb838045bc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:23.694453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b525adfe144b26ae9608eb838045bc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:23.696930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b525adfe144b26ae9608eb838045bc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:23.697581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b525adfe144b26ae9608eb838045bc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:23.763601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b525adfe144b26ae9608eb838045bc'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:23.777891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 999/2020] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:23 2022] DELETE /identity/v3/users/98b525adfe144b26ae9608eb838045bc => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:23.782194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be08f-e246-4669-8ca7-f64ddd67c9d0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:23.782631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be08f-e246-4669-8ca7-f64ddd67c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:23.782757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be08f-e246-4669-8ca7-f64ddd67c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:23.822117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-716be08f-e246-4669-8ca7-f64ddd67c9d0 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:23.823234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-716be08f-e246-4669-8ca7-f64ddd67c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:23.883828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1022/2021] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:23.893477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:23.894917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:23.958584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:23.973374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ae8b5-5f2e-4ed1-b7c6-35547ec4bec5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:23.980222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59f667-e2a4-4def-ae05-edc930b9e3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:23.980222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59f667-e2a4-4def-ae05-edc930b9e3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:23.980222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59f667-e2a4-4def-ae05-edc930b9e3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:23.980222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59f667-e2a4-4def-ae05-edc930b9e3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:23.986997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59f667-e2a4-4def-ae05-edc930b9e3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:23.989290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59f667-e2a4-4def-ae05-edc930b9e3c2 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:23.998496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1000/2022] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:24.417079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be08f-e246-4669-8ca7-f64ddd67c9d0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:24.418546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-716be08f-e246-4669-8ca7-f64ddd67c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:24.430277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be08f-e246-4669-8ca7-f64ddd67c9d0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:24.435084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:24.435943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:24.436222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:24.436497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] PATH_INFO: `/v3/projects/7078de2623d2496991b394867371614b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:24.437711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7078de2623d2496991b394867371614b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:24.439623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:24.605411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7078de2623d2496991b394867371614b', 'request_id': 'req-e2c50801-1043-4336-9497-244cc258a489'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.611830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7078de2623d2496991b394867371614b', 'request_id': 'req-e2c50801-1043-4336-9497-244cc258a489'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.618213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7078de2623d2496991b394867371614b', 'request_id': 'req-e2c50801-1043-4336-9497-244cc258a489'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.707449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] The token cache is being invalidate because project 7078de2623d2496991b394867371614b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:24.707730 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.708427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.710009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1023/2023] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:24 2022] DELETE /identity/v3/projects/7078de2623d2496991b394867371614b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:24.713914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59f667-e2a4-4def-ae05-edc930b9e3c2 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:24.723453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59f667-e2a4-4def-ae05-edc930b9e3c2 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:24.782039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:24.782917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:24.783319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:24.783580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] PATH_INFO: `/v3/users/6f93be4a54ec44f8836d541adfbe3a65` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:24.784864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f93be4a54ec44f8836d541adfbe3a65)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:24.786782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:24.972051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f93be4a54ec44f8836d541adfbe3a65'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.984827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f93be4a54ec44f8836d541adfbe3a65'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.989597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f93be4a54ec44f8836d541adfbe3a65'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.989597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f93be4a54ec44f8836d541adfbe3a65'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:24.991442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:24.992988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:25.006362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c50801-1043-4336-9497-244cc258a489 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:25.010909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:25.011764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:25.012046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:25.012351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] PATH_INFO: `/v3/users/9cbe6758fa024de5af49b49e340618aa` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:25.013568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cbe6758fa024de5af49b49e340618aa)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:25.015483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:25.101057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-e2d8a9fb-a66e-4b0e-bf64-73c19d999438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f93be4a54ec44f8836d541adfbe3a65'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.114925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1001/2024] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:24 2022] DELETE /identity/v3/users/6f93be4a54ec44f8836d541adfbe3a65 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:25.217416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cbe6758fa024de5af49b49e340618aa'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.224210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cbe6758fa024de5af49b49e340618aa'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.226030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:25.235942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cbe6758fa024de5af49b49e340618aa'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.236660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cbe6758fa024de5af49b49e340618aa'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.375713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cbe6758fa024de5af49b49e340618aa'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.385535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1024/2025] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:24 2022] DELETE /identity/v3/users/9cbe6758fa024de5af49b49e340618aa => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:25.392650 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc6b7b-d784-4aa3-8633-1f9eb4ed20d3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:25.393020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc6b7b-d784-4aa3-8633-1f9eb4ed20d3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:25.393442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc6b7b-d784-4aa3-8633-1f9eb4ed20d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:25.413452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-83fc6b7b-d784-4aa3-8633-1f9eb4ed20d3 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:25.419356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-83fc6b7b-d784-4aa3-8633-1f9eb4ed20d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:25.492491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1002/2026] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:25.502029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:25.503506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:25.561568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019b8de-2522-4219-a016-739117d698e4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:25.565502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b730b-a9d3-4dc6-b5dc-c366d83d7060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:25.566153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b730b-a9d3-4dc6-b5dc-c366d83d7060 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:25.566437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b730b-a9d3-4dc6-b5dc-c366d83d7060 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:25.566713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b730b-a9d3-4dc6-b5dc-c366d83d7060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:25.576540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc6b7b-d784-4aa3-8633-1f9eb4ed20d3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:25.576540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-83fc6b7b-d784-4aa3-8633-1f9eb4ed20d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:25.579316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b730b-a9d3-4dc6-b5dc-c366d83d7060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:25.581026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b730b-a9d3-4dc6-b5dc-c366d83d7060 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:25.582625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:25.589446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc6b7b-d784-4aa3-8633-1f9eb4ed20d3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:25.593538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1025/2027] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:25.594807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:25.595722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:25.608968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:25.608968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] PATH_INFO: `/v3/projects/cff5d8a1684949ea894fef882beffe5d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:25.608968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff5d8a1684949ea894fef882beffe5d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:25.608968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:25.714248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff5d8a1684949ea894fef882beffe5d', 'request_id': 'req-53c9bd9f-a76b-4f9e-b532-ac86d826329a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.720527 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff5d8a1684949ea894fef882beffe5d', 'request_id': 'req-53c9bd9f-a76b-4f9e-b532-ac86d826329a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.731716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff5d8a1684949ea894fef882beffe5d', 'request_id': 'req-53c9bd9f-a76b-4f9e-b532-ac86d826329a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.924497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] The token cache is being invalidate because project cff5d8a1684949ea894fef882beffe5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:25.924863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.925765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:25.927466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1003/2028] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:25 2022] DELETE /identity/v3/projects/cff5d8a1684949ea894fef882beffe5d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:25.932221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f728772-35b6-458a-ba6f-d42b93c7f2b0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:25.932519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f728772-35b6-458a-ba6f-d42b93c7f2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:25.932808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f728772-35b6-458a-ba6f-d42b93c7f2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:25.950447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3f728772-35b6-458a-ba6f-d42b93c7f2b0 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:25.951646 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3f728772-35b6-458a-ba6f-d42b93c7f2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.020073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1026/2029] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:26.028537 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.029590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.030370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650c4a-2d8f-4d1f-af05-2f6057614083 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.030966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650c4a-2d8f-4d1f-af05-2f6057614083 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.031535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39650c4a-2d8f-4d1f-af05-2f6057614083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.045973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9bd9f-a76b-4f9e-b532-ac86d826329a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.052092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.053180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.053473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.054233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] PATH_INFO: `/v3/projects/c877d4e75dbc4bdbb9fd24cbc78c75b0` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.055600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c877d4e75dbc4bdbb9fd24cbc78c75b0)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.058247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-39650c4a-2d8f-4d1f-af05-2f6057614083 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:26.062681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.062994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-39650c4a-2d8f-4d1f-af05-2f6057614083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.070334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1027/2030] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:26.075281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39650c4a-2d8f-4d1f-af05-2f6057614083 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.087476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39650c4a-2d8f-4d1f-af05-2f6057614083 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.092542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.093234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.093523 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.093812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.094528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:26.094839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.096446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.112221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1028/2031] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:26.113649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.123264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb977318-f9cf-428b-842b-70a93e928033 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.127950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.129135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.129425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.129698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.130395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.132274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.172932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c877d4e75dbc4bdbb9fd24cbc78c75b0', 'request_id': 'req-ceb8e451-c02d-416c-ac81-008579ec2a13'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.180559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c877d4e75dbc4bdbb9fd24cbc78c75b0', 'request_id': 'req-ceb8e451-c02d-416c-ac81-008579ec2a13'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.187308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c877d4e75dbc4bdbb9fd24cbc78c75b0', 'request_id': 'req-ceb8e451-c02d-416c-ac81-008579ec2a13'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.214521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1029/2032] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:26.214521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.214521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.253739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:26.276075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b6375-ea49-4b2c-abac-0d50803ea0a2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.280825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.281693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.281978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.282256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] PATH_INFO: `/v3/users/f65c64f45b514865a1b46317c6519603` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.283087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] The token cache is being invalidate because project c877d4e75dbc4bdbb9fd24cbc78c75b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:26.283384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.283500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65c64f45b514865a1b46317c6519603)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.284008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.285446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.285717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1004/2033] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:26 2022] DELETE /identity/v3/projects/c877d4e75dbc4bdbb9fd24cbc78c75b0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:26.286642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.287953 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.352024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8e451-c02d-416c-ac81-008579ec2a13 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.356692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122780f7-a9e6-4bac-9ea0-ad3900bbbf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.357439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122780f7-a9e6-4bac-9ea0-ad3900bbbf36 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.357722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122780f7-a9e6-4bac-9ea0-ad3900bbbf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.357999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122780f7-a9e6-4bac-9ea0-ad3900bbbf36 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.358716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122780f7-a9e6-4bac-9ea0-ad3900bbbf36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.360661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122780f7-a9e6-4bac-9ea0-ad3900bbbf36 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.472966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:26.479032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1005/2034] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:26.480266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7e749-604d-4697-b849-89b783b879d5 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.480553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7e749-604d-4697-b849-89b783b879d5 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.480831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7e749-604d-4697-b849-89b783b879d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.500468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-85e7e749-604d-4697-b849-89b783b879d5 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:26.501648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-85e7e749-604d-4697-b849-89b783b879d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.517375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1006/2035] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:26.517375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7e749-604d-4697-b849-89b783b879d5 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.517738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65c64f45b514865a1b46317c6519603'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.525412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7e749-604d-4697-b849-89b783b879d5 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.530640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69e971-df65-4569-a5cc-09e872f22052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.531406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69e971-df65-4569-a5cc-09e872f22052 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.531770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69e971-df65-4569-a5cc-09e872f22052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.532119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69e971-df65-4569-a5cc-09e872f22052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.532228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65c64f45b514865a1b46317c6519603'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.532990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69e971-df65-4569-a5cc-09e872f22052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.534780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69e971-df65-4569-a5cc-09e872f22052 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.549325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1007/2036] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:26.549325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ee604-dc1a-4d43-8094-ca7294d57452 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.549325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ee604-dc1a-4d43-8094-ca7294d57452 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.549325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ee604-dc1a-4d43-8094-ca7294d57452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.568817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65c64f45b514865a1b46317c6519603'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.569654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65c64f45b514865a1b46317c6519603'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.574040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d55ee604-dc1a-4d43-8094-ca7294d57452 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:26.574040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d55ee604-dc1a-4d43-8094-ca7294d57452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.584698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1008/2037] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:26.585878 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ee604-dc1a-4d43-8094-ca7294d57452 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.595562 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ee604-dc1a-4d43-8094-ca7294d57452 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.602503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a78dc2-cf6e-48df-8105-67c81b0931d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.603252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a78dc2-cf6e-48df-8105-67c81b0931d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.603603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a78dc2-cf6e-48df-8105-67c81b0931d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.604580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a78dc2-cf6e-48df-8105-67c81b0931d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.605416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a78dc2-cf6e-48df-8105-67c81b0931d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.607099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a78dc2-cf6e-48df-8105-67c81b0931d0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.619088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1009/2038] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:26.620087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a78dc2-cf6e-48df-8105-67c81b0931d0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.633002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65c64f45b514865a1b46317c6519603'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:26.638468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a78dc2-cf6e-48df-8105-67c81b0931d0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.643245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1030/2039] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:26 2022] DELETE /identity/v3/users/f65c64f45b514865a1b46317c6519603 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:26.643245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.643245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.716662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f14134-4087-4ba5-894f-acd8d6c0964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.717481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14134-4087-4ba5-894f-acd8d6c0964a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.717822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14134-4087-4ba5-894f-acd8d6c0964a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.718127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14134-4087-4ba5-894f-acd8d6c0964a admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.718891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f14134-4087-4ba5-894f-acd8d6c0964a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:26.719202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f14134-4087-4ba5-894f-acd8d6c0964a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.720822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f14134-4087-4ba5-894f-acd8d6c0964a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.738049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1010/2040] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:26.739554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4dab-9304-43d4-8cc6-efcc6e35d2fb None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.741976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4dab-9304-43d4-8cc6-efcc6e35d2fb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.741976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce4dab-9304-43d4-8cc6-efcc6e35d2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.744449 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcfb7b-b669-4805-a68f-8f5c9c84e3d8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.749502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.750464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.750817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.751226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] PATH_INFO: `/v3/projects/b24fb71f54464b39b35f4b3ff92736d8/users/6b1f7ccdc4b94408b5b32374a7572415/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.761646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ccce4dab-9304-43d4-8cc6-efcc6e35d2fb None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:26.762966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ccce4dab-9304-43d4-8cc6-efcc6e35d2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.769651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b24fb71f54464b39b35f4b3ff92736d8, user_id=6b1f7ccdc4b94408b5b32374a7572415, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.772823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.775260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1011/2041] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:26.776285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce4dab-9304-43d4-8cc6-efcc6e35d2fb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.777833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ccce4dab-9304-43d4-8cc6-efcc6e35d2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.790214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce4dab-9304-43d4-8cc6-efcc6e35d2fb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.795755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531af855-a94a-4bfc-8d9b-7dc0557f800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.796433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531af855-a94a-4bfc-8d9b-7dc0557f800a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.796749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531af855-a94a-4bfc-8d9b-7dc0557f800a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.797034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531af855-a94a-4bfc-8d9b-7dc0557f800a admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.797784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531af855-a94a-4bfc-8d9b-7dc0557f800a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:26.799734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531af855-a94a-4bfc-8d9b-7dc0557f800a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:26.839051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1031/2042] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:26 2022] PUT /identity/v3/projects/b24fb71f54464b39b35f4b3ff92736d8/users/6b1f7ccdc4b94408b5b32374a7572415/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:26.840578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:26.842183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.880181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1012/2043] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:26.881544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c1295-794e-4e8b-bddd-372925e2d590 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.881829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c1295-794e-4e8b-bddd-372925e2d590 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.882106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c1295-794e-4e8b-bddd-372925e2d590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.885362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:26.899737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0148e-7bba-4494-9da4-e01759edf6e5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:26.914439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-994c1295-794e-4e8b-bddd-372925e2d590 None None] MFA Rules not processed for user `6b1f7ccdc4b94408b5b32374a7572415`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:26.915717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-994c1295-794e-4e8b-bddd-372925e2d590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:26.953522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b93ae5-4aee-4c82-93b0-1ad7ae456edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:26.954166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b93ae5-4aee-4c82-93b0-1ad7ae456edc service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:26.954441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b93ae5-4aee-4c82-93b0-1ad7ae456edc service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:26.954721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b93ae5-4aee-4c82-93b0-1ad7ae456edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:26.980974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1013/2044] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:26.984630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c1295-794e-4e8b-bddd-372925e2d590 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.000834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:27.016895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b93ae5-4aee-4c82-93b0-1ad7ae456edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.018447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b93ae5-4aee-4c82-93b0-1ad7ae456edc service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.028931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1032/2045] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:27.029829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b93ae5-4aee-4c82-93b0-1ad7ae456edc service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.062057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b93ae5-4aee-4c82-93b0-1ad7ae456edc service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.067554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f8e0d-b391-4927-a991-4f57fadecb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.068234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f8e0d-b391-4927-a991-4f57fadecb9c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.068505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f8e0d-b391-4927-a991-4f57fadecb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.068788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f8e0d-b391-4927-a991-4f57fadecb9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.068899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c1295-794e-4e8b-bddd-372925e2d590 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.069521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f8e0d-b391-4927-a991-4f57fadecb9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.071431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f8e0d-b391-4927-a991-4f57fadecb9c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.073794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4b724-0f32-4d5b-8e91-8c05a21c6f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.074468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b724-0f32-4d5b-8e91-8c05a21c6f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.074788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b724-0f32-4d5b-8e91-8c05a21c6f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.075022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b724-0f32-4d5b-8e91-8c05a21c6f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.075731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4b724-0f32-4d5b-8e91-8c05a21c6f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.078040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4b724-0f32-4d5b-8e91-8c05a21c6f2b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.218390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1014/2046] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:26 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:27.218592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1033/2047] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:27 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:27.219573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4b724-0f32-4d5b-8e91-8c05a21c6f2b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.220806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f8e0d-b391-4927-a991-4f57fadecb9c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.229540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4b724-0f32-4d5b-8e91-8c05a21c6f2b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.294063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f8e0d-b391-4927-a991-4f57fadecb9c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.304750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea55d3-ec89-4609-b606-09f0cd61276c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.304750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea55d3-ec89-4609-b606-09f0cd61276c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.304750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea55d3-ec89-4609-b606-09f0cd61276c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.304750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea55d3-ec89-4609-b606-09f0cd61276c admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.305090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea55d3-ec89-4609-b606-09f0cd61276c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.307402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea55d3-ec89-4609-b606-09f0cd61276c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.319043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273e8f8-e706-46f6-8800-d3b4c349e5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.320009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e8f8-e706-46f6-8800-d3b4c349e5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.320299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e8f8-e706-46f6-8800-d3b4c349e5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.320582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e8f8-e706-46f6-8800-d3b4c349e5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.321361 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273e8f8-e706-46f6-8800-d3b4c349e5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.323149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a273e8f8-e706-46f6-8800-d3b4c349e5f9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.336544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1015/2048] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:27.337993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273e8f8-e706-46f6-8800-d3b4c349e5f9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.347487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273e8f8-e706-46f6-8800-d3b4c349e5f9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.352247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d14e6-a060-4d7c-bdca-bfc221fd06dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.352972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d14e6-a060-4d7c-bdca-bfc221fd06dc admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.353382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d14e6-a060-4d7c-bdca-bfc221fd06dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.353729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d14e6-a060-4d7c-bdca-bfc221fd06dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.354529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d14e6-a060-4d7c-bdca-bfc221fd06dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.356210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d14e6-a060-4d7c-bdca-bfc221fd06dc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.367058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1016/2049] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:27.370928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d14e6-a060-4d7c-bdca-bfc221fd06dc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.381225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d14e6-a060-4d7c-bdca-bfc221fd06dc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.386468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661449ba-aff6-40e1-9f1c-2574bc34f4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.390558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661449ba-aff6-40e1-9f1c-2574bc34f4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.390558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661449ba-aff6-40e1-9f1c-2574bc34f4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.390558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661449ba-aff6-40e1-9f1c-2574bc34f4d2 admin admin] PATH_INFO: `/v3/projects/8feafd08250a444da0fc8b51cfcd22c8/users/e514c31512ba40a38e82d73c381b0246/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.405836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661449ba-aff6-40e1-9f1c-2574bc34f4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8feafd08250a444da0fc8b51cfcd22c8, user_id=e514c31512ba40a38e82d73c381b0246, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.408671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661449ba-aff6-40e1-9f1c-2574bc34f4d2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.410969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:27.417830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1034/2050] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:27 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:27.421825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea55d3-ec89-4609-b606-09f0cd61276c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.431874 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea55d3-ec89-4609-b606-09f0cd61276c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.436550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283e2aa-b3dc-4056-a942-6ba768976806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.442053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283e2aa-b3dc-4056-a942-6ba768976806 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.442053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283e2aa-b3dc-4056-a942-6ba768976806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.442053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283e2aa-b3dc-4056-a942-6ba768976806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.442053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283e2aa-b3dc-4056-a942-6ba768976806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.442053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283e2aa-b3dc-4056-a942-6ba768976806 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.451094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1035/2051] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:27.455057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283e2aa-b3dc-4056-a942-6ba768976806 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.464421 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283e2aa-b3dc-4056-a942-6ba768976806 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.469321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.470209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.470496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.470825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] PATH_INFO: `/v3/projects/5c6eda9adb3445c7beba0f0c4e5a740b/users/aaae9156704f4bab92251443f1ef4873/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.485041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6eda9adb3445c7beba0f0c4e5a740b, user_id=aaae9156704f4bab92251443f1ef4873, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.488532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.535284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1017/2052] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:27 2022] PUT /identity/v3/projects/8feafd08250a444da0fc8b51cfcd22c8/users/e514c31512ba40a38e82d73c381b0246/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:27.546430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff606279-6adc-4552-bb4e-f2fa7d3a997c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.546720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff606279-6adc-4552-bb4e-f2fa7d3a997c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.546999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff606279-6adc-4552-bb4e-f2fa7d3a997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.573549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ff606279-6adc-4552-bb4e-f2fa7d3a997c None None] MFA Rules not processed for user `e514c31512ba40a38e82d73c381b0246`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:27.574766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ff606279-6adc-4552-bb4e-f2fa7d3a997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:27.595429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1036/2053] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:27 2022] PUT /identity/v3/projects/5c6eda9adb3445c7beba0f0c4e5a740b/users/aaae9156704f4bab92251443f1ef4873/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:27.595429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.595429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:27.625847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:27.651025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1018/2054] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:27.651920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff606279-6adc-4552-bb4e-f2fa7d3a997c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.676117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39165b-9928-4e46-aa03-21bfb3382bbc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.680747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.681629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.681930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.682186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] PATH_INFO: `/v3/projects/a932c0c29ac746918db9d77537693066` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.683352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a932c0c29ac746918db9d77537693066)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.685314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.714996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff606279-6adc-4552-bb4e-f2fa7d3a997c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.720039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22785f-28ec-45a7-845a-fcd7fb82f4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.720704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22785f-28ec-45a7-845a-fcd7fb82f4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.720986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22785f-28ec-45a7-845a-fcd7fb82f4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.721337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22785f-28ec-45a7-845a-fcd7fb82f4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.722207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22785f-28ec-45a7-845a-fcd7fb82f4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.723728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22785f-28ec-45a7-845a-fcd7fb82f4a9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.734376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1019/2055] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:27.736826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22785f-28ec-45a7-845a-fcd7fb82f4a9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.746756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22785f-28ec-45a7-845a-fcd7fb82f4a9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.752674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6766fcf-0748-4110-bd8d-8934baf42e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.753924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6766fcf-0748-4110-bd8d-8934baf42e40 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.754208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6766fcf-0748-4110-bd8d-8934baf42e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.754481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6766fcf-0748-4110-bd8d-8934baf42e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.755157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6766fcf-0748-4110-bd8d-8934baf42e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.757092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6766fcf-0748-4110-bd8d-8934baf42e40 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.804354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a932c0c29ac746918db9d77537693066', 'request_id': 'req-337b1f70-dcdb-4701-aa6b-f5bf905f7653'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:27.810637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a932c0c29ac746918db9d77537693066', 'request_id': 'req-337b1f70-dcdb-4701-aa6b-f5bf905f7653'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:27.816525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a932c0c29ac746918db9d77537693066', 'request_id': 'req-337b1f70-dcdb-4701-aa6b-f5bf905f7653'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:27.840492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1020/2056] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:27 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:27.845034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6766fcf-0748-4110-bd8d-8934baf42e40 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.886622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] The token cache is being invalidate because project a932c0c29ac746918db9d77537693066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:27.886915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:27.887582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:27.889212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1037/2057] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:27 2022] DELETE /identity/v3/projects/a932c0c29ac746918db9d77537693066 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:27.889960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.891660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:27.924387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6766fcf-0748-4110-bd8d-8934baf42e40 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:27.929675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:27.930638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:27.930991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:27.931336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] PATH_INFO: `/v3/projects/5c6eda9adb3445c7beba0f0c4e5a740b/users/aaae9156704f4bab92251443f1ef4873/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:27.933480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6eda9adb3445c7beba0f0c4e5a740b, user_id=aaae9156704f4bab92251443f1ef4873, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:27.936262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:27.949647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1021/2058] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:27 2022] PUT /identity/v3/projects/5c6eda9adb3445c7beba0f0c4e5a740b/users/aaae9156704f4bab92251443f1ef4873/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:27.950507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:27.953603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:27.960230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:27.972513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b1f70-dcdb-4701-aa6b-f5bf905f7653 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:28.013393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:28.031022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8df19-7866-4ede-a31e-065c26e87e91 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:28.035747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d41c6-2e0e-4878-a28e-d36728acd712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:28.036444 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:28.037001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.037482 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d41c6-2e0e-4878-a28e-d36728acd712 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.037774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d41c6-2e0e-4878-a28e-d36728acd712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.038048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d41c6-2e0e-4878-a28e-d36728acd712 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.038998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d41c6-2e0e-4878-a28e-d36728acd712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:28.041203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d41c6-2e0e-4878-a28e-d36728acd712 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:28.045431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.045888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] PATH_INFO: `/v3/users/002873ceedf346dea8c4a166022f946c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.047522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002873ceedf346dea8c4a166022f946c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:28.049681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:28.121931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1038/2059] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:27 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:28.121931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c0c0-824c-4acd-9d14-42f02bcb93c3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.121931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c0c0-824c-4acd-9d14-42f02bcb93c3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.121931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c0c0-824c-4acd-9d14-42f02bcb93c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.146872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d1a7c0c0-824c-4acd-9d14-42f02bcb93c3 None None] MFA Rules not processed for user `aaae9156704f4bab92251443f1ef4873`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:28.148155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d1a7c0c0-824c-4acd-9d14-42f02bcb93c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:28.221087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1039/2060] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:28.222593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7c0c0-824c-4acd-9d14-42f02bcb93c3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:28.232741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7c0c0-824c-4acd-9d14-42f02bcb93c3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:28.237698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:28.240776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.240776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.240776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.240776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:28.241339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:28.251151 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1040/2061] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:28.251909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:28.253572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:28.280295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002873ceedf346dea8c4a166022f946c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:28.294024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002873ceedf346dea8c4a166022f946c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:28.298551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002873ceedf346dea8c4a166022f946c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:28.298551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002873ceedf346dea8c4a166022f946c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:28.314025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c29a-810d-4565-8775-cbc5e2633216 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:28.318505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:28.319156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.319441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.319721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.321603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:28.334607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:28.336052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:28.345559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1041/2062] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:28.346413 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:28.347739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:28.360278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344bad7-d42f-4ca9-b58d-b2ec9c2962b5 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:28.365333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a185564-5a7d-4f97-8735-6e52d1570994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:28.367006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a185564-5a7d-4f97-8735-6e52d1570994 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.367006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a185564-5a7d-4f97-8735-6e52d1570994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.367006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a185564-5a7d-4f97-8735-6e52d1570994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.367342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a185564-5a7d-4f97-8735-6e52d1570994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:28.368844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a185564-5a7d-4f97-8735-6e52d1570994 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:28.370188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:28.380041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1042/2063] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:28.384040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a185564-5a7d-4f97-8735-6e52d1570994 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:28.387192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d722f38a-cd6e-4afe-a97f-39e0255634c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002873ceedf346dea8c4a166022f946c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:28.394434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a185564-5a7d-4f97-8735-6e52d1570994 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:28.401502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23209b7d-484a-49f7-8db8-6c91f457f941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:28.401502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23209b7d-484a-49f7-8db8-6c91f457f941 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.401502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23209b7d-484a-49f7-8db8-6c91f457f941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.401502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23209b7d-484a-49f7-8db8-6c91f457f941 admin admin] PATH_INFO: `/v3/projects/56322bc8c6a14de498f19d610dc8e1ef/users/04a58755751845028093f563d636ee57/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.402023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1022/2064] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:27 2022] DELETE /identity/v3/users/002873ceedf346dea8c4a166022f946c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:28.415996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23209b7d-484a-49f7-8db8-6c91f457f941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56322bc8c6a14de498f19d610dc8e1ef, user_id=04a58755751845028093f563d636ee57, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:28.418681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23209b7d-484a-49f7-8db8-6c91f457f941 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:28.554860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1043/2065] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:28 2022] PUT /identity/v3/projects/56322bc8c6a14de498f19d610dc8e1ef/users/04a58755751845028093f563d636ee57/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:28.571329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dda541-968d-4d87-97a1-d5213b852d08 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.571632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dda541-968d-4d87-97a1-d5213b852d08 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.571919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dda541-968d-4d87-97a1-d5213b852d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.607480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-35dda541-968d-4d87-97a1-d5213b852d08 None None] MFA Rules not processed for user `04a58755751845028093f563d636ee57`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:28.621936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-35dda541-968d-4d87-97a1-d5213b852d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:28.777138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1023/2066] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:28.782184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23209b7d-484a-49f7-8db8-6c91f457f941 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:28.855412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23209b7d-484a-49f7-8db8-6c91f457f941 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:28.860322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d60fe-33af-4f71-b111-2d52afe4fb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:28.861018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d60fe-33af-4f71-b111-2d52afe4fb17 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.861338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d60fe-33af-4f71-b111-2d52afe4fb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.861622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d60fe-33af-4f71-b111-2d52afe4fb17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.862339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d60fe-33af-4f71-b111-2d52afe4fb17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:28.865030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d60fe-33af-4f71-b111-2d52afe4fb17 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:28.886500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dda541-968d-4d87-97a1-d5213b852d08 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:28.888452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-35dda541-968d-4d87-97a1-d5213b852d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:28.906627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dda541-968d-4d87-97a1-d5213b852d08 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:28.912617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:28.922274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:28.922566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:28.922854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] PATH_INFO: `/v3/projects/85945bd0af2447968d4c56eb14229f4e` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:28.924373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85945bd0af2447968d4c56eb14229f4e)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:28.926978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:28.966405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1044/2067] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:28 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:28.974796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d60fe-33af-4f71-b111-2d52afe4fb17 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.041086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85945bd0af2447968d4c56eb14229f4e', 'request_id': 'req-4e926166-730b-42ac-8d31-0ac951523c29'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.043540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d60fe-33af-4f71-b111-2d52afe4fb17 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.044023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:29.049264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85945bd0af2447968d4c56eb14229f4e', 'request_id': 'req-4e926166-730b-42ac-8d31-0ac951523c29'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.059628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85945bd0af2447968d4c56eb14229f4e', 'request_id': 'req-4e926166-730b-42ac-8d31-0ac951523c29'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.107995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6a579-90ca-4324-88e8-7776ede64be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.109276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6a579-90ca-4324-88e8-7776ede64be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.109566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6a579-90ca-4324-88e8-7776ede64be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.109847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6a579-90ca-4324-88e8-7776ede64be7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.110626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6a579-90ca-4324-88e8-7776ede64be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.113370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6a579-90ca-4324-88e8-7776ede64be7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.122084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:29.129165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] The token cache is being invalidate because project 85945bd0af2447968d4c56eb14229f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:29.132760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.132760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.132760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1024/2068] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:28 2022] DELETE /identity/v3/projects/85945bd0af2447968d4c56eb14229f4e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:29.133009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.134165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.147090 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e926166-730b-42ac-8d31-0ac951523c29 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.152496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.153450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.153728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.154006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] PATH_INFO: `/v3/users/6b1f7ccdc4b94408b5b32374a7572415` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.158319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b1f7ccdc4b94408b5b32374a7572415)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.158319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.210102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1045/2069] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:28 2022] POST /identity/v3/users => generated 295 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:29.211801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a3c4-3c08-4b3e-b517-582c03704bc4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.212092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a3c4-3c08-4b3e-b517-582c03704bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.212367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a3c4-3c08-4b3e-b517-582c03704bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.260029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7169a3c4-3c08-4b3e-b517-582c03704bc4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:29.261276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7169a3c4-3c08-4b3e-b517-582c03704bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.269202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1046/2070] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:29.269979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169a3c4-3c08-4b3e-b517-582c03704bc4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.271239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7169a3c4-3c08-4b3e-b517-582c03704bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.282411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169a3c4-3c08-4b3e-b517-582c03704bc4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.287722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58ced6-a8c3-47c8-a67a-70039e9b78f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.288381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ced6-a8c3-47c8-a67a-70039e9b78f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.288660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ced6-a8c3-47c8-a67a-70039e9b78f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.288938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ced6-a8c3-47c8-a67a-70039e9b78f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.289700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58ced6-a8c3-47c8-a67a-70039e9b78f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.291285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58ced6-a8c3-47c8-a67a-70039e9b78f1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.303511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1047/2071] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:29.304819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fdbcb-eb14-4803-b803-c5024c984727 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.305168 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fdbcb-eb14-4803-b803-c5024c984727 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.305495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fdbcb-eb14-4803-b803-c5024c984727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.323407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-cf1fdbcb-eb14-4803-b803-c5024c984727 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:29.324586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cf1fdbcb-eb14-4803-b803-c5024c984727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.333499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1048/2072] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:29.337291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fdbcb-eb14-4803-b803-c5024c984727 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.346555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fdbcb-eb14-4803-b803-c5024c984727 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.351714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.352587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.352874 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.353179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] PATH_INFO: `/v3/projects/8f65e17dbea549b29dcd9e9263c54151/users/9734245a099444c098cf25bececdd637/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.370401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f65e17dbea549b29dcd9e9263c54151, user_id=9734245a099444c098cf25bececdd637, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.371240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b1f7ccdc4b94408b5b32374a7572415'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.373141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.384578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b1f7ccdc4b94408b5b32374a7572415'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.385243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b1f7ccdc4b94408b5b32374a7572415'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.386282 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b1f7ccdc4b94408b5b32374a7572415'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.449513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1049/2073] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:29 2022] PUT /identity/v3/projects/8f65e17dbea549b29dcd9e9263c54151/users/9734245a099444c098cf25bececdd637/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:29.450337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.451876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.455247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b1f7ccdc4b94408b5b32374a7572415'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:29.470760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1025/2074] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:29 2022] DELETE /identity/v3/users/6b1f7ccdc4b94408b5b32374a7572415 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:29.472672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.474119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.520422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1462c7-4824-4242-92fc-f4ba19a62b4c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.546720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a3ee8-1a9b-4515-bedc-204f179f819b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.552205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac122f58-9113-4f7c-ab43-45e5229cafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.561020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac122f58-9113-4f7c-ab43-45e5229cafed admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.561020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac122f58-9113-4f7c-ab43-45e5229cafed admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.561020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac122f58-9113-4f7c-ab43-45e5229cafed admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.561020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac122f58-9113-4f7c-ab43-45e5229cafed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.561020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac122f58-9113-4f7c-ab43-45e5229cafed admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.566174 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1026/2075] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:29.567468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda43eea-756e-4e94-ba03-1d2afaa45b51 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.567803 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda43eea-756e-4e94-ba03-1d2afaa45b51 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.568089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda43eea-756e-4e94-ba03-1d2afaa45b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.577716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.578382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.578668 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.578947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.579682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:29.579995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.582917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.586204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dda43eea-756e-4e94-ba03-1d2afaa45b51 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:29.587433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dda43eea-756e-4e94-ba03-1d2afaa45b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.595153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1050/2076] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:29.595283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1027/2077] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:29.596200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.599936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda43eea-756e-4e94-ba03-1d2afaa45b51 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.607815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827de6d0-b3db-4362-ac7e-e5b35aa70d8e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.609764 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda43eea-756e-4e94-ba03-1d2afaa45b51 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.613310 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.614404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.614694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.614853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbacca-c417-481b-925f-e6cc6c657ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.614962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] PATH_INFO: `/v3/projects/8f65e17dbea549b29dcd9e9263c54151/users/9734245a099444c098cf25bececdd637/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.615478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbacca-c417-481b-925f-e6cc6c657ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.615768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbacca-c417-481b-925f-e6cc6c657ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.616081 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbacca-c417-481b-925f-e6cc6c657ab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.616926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbacca-c417-481b-925f-e6cc6c657ab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.617134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f65e17dbea549b29dcd9e9263c54151, user_id=9734245a099444c098cf25bececdd637, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.618888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbacca-c417-481b-925f-e6cc6c657ab0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.619895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.631932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1051/2078] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:29 2022] PUT /identity/v3/projects/8f65e17dbea549b29dcd9e9263c54151/users/9734245a099444c098cf25bececdd637/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:29.632832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.634687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.653223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:29.717631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a22a1d-340d-4046-9129-cf1651a720d4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.721756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c75070-f3b8-4114-a7a1-bf6b683f9ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.722969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c75070-f3b8-4114-a7a1-bf6b683f9ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.723255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c75070-f3b8-4114-a7a1-bf6b683f9ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.723533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c75070-f3b8-4114-a7a1-bf6b683f9ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.748088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1028/2079] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:29 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:29.750661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82076900-3540-439c-a512-ce037210ee87 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.750661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82076900-3540-439c-a512-ce037210ee87 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.750661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82076900-3540-439c-a512-ce037210ee87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.775947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-82076900-3540-439c-a512-ce037210ee87 None None] MFA Rules not processed for user `9734245a099444c098cf25bececdd637`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:29.777190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-82076900-3540-439c-a512-ce037210ee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.785834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:29.788615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c75070-f3b8-4114-a7a1-bf6b683f9ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.790212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c75070-f3b8-4114-a7a1-bf6b683f9ac4 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.799756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1052/2080] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:29.800547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c75070-f3b8-4114-a7a1-bf6b683f9ac4 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.811554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c75070-f3b8-4114-a7a1-bf6b683f9ac4 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.892959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1029/2081] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:29.898982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494bf21-5bb3-4b7a-b60d-7413e7740e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.902050 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494bf21-5bb3-4b7a-b60d-7413e7740e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.902489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494bf21-5bb3-4b7a-b60d-7413e7740e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.903270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494bf21-5bb3-4b7a-b60d-7413e7740e8c admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.903722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494bf21-5bb3-4b7a-b60d-7413e7740e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:29.906046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494bf21-5bb3-4b7a-b60d-7413e7740e8c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:29.910667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82076900-3540-439c-a512-ce037210ee87 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:29.910667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-82076900-3540-439c-a512-ce037210ee87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:29.973024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82076900-3540-439c-a512-ce037210ee87 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:29.976770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:29.977453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:29.977741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:29.978023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:29.979833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:30.000748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1053/2082] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:29 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:30.001806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494bf21-5bb3-4b7a-b60d-7413e7740e8c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.013720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494bf21-5bb3-4b7a-b60d-7413e7740e8c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.020256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3df9b-bee2-4fd0-b5ee-ffb0851d3665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.021053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3df9b-bee2-4fd0-b5ee-ffb0851d3665 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.021448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3df9b-bee2-4fd0-b5ee-ffb0851d3665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.021797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b3df9b-bee2-4fd0-b5ee-ffb0851d3665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.022713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3df9b-bee2-4fd0-b5ee-ffb0851d3665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.024451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b3df9b-bee2-4fd0-b5ee-ffb0851d3665 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.041244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1054/2083] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:30.042218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3df9b-bee2-4fd0-b5ee-ffb0851d3665 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.050669 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.052112 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.052355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b3df9b-bee2-4fd0-b5ee-ffb0851d3665 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.059031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7107111-31c0-48d9-8d0d-04c9ed503ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.059031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7107111-31c0-48d9-8d0d-04c9ed503ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.059305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7107111-31c0-48d9-8d0d-04c9ed503ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.059876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7107111-31c0-48d9-8d0d-04c9ed503ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.060612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7107111-31c0-48d9-8d0d-04c9ed503ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.064671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7107111-31c0-48d9-8d0d-04c9ed503ba9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.064759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1030/2084] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:30.076244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1055/2085] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:30.080344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.092184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a938-0db8-4553-aec4-03fc6c675aec service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.097721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9ec9-2751-4e57-856c-f00c4bfe7b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.098325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9ec9-2751-4e57-856c-f00c4bfe7b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.098639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9ec9-2751-4e57-856c-f00c4bfe7b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.098976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9ec9-2751-4e57-856c-f00c4bfe7b26 admin admin] PATH_INFO: `/v3/projects/b8f3144aed5c4c64a2d723dfc09db682/users/9e4a3b3acd8345388faf58d236040872/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.117531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada9ec9-2751-4e57-856c-f00c4bfe7b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f3144aed5c4c64a2d723dfc09db682, user_id=9e4a3b3acd8345388faf58d236040872, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.121956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada9ec9-2751-4e57-856c-f00c4bfe7b26 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.226116 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1031/2086] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:30 2022] PUT /identity/v3/projects/b8f3144aed5c4c64a2d723dfc09db682/users/9e4a3b3acd8345388faf58d236040872/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:30.238188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b110e-0de5-47f4-b6e9-b34709ee1304 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.238496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b110e-0de5-47f4-b6e9-b34709ee1304 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.238776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b110e-0de5-47f4-b6e9-b34709ee1304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.256689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:30.265271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2a0b110e-0de5-47f4-b6e9-b34709ee1304 None None] MFA Rules not processed for user `9e4a3b3acd8345388faf58d236040872`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:30.266470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2a0b110e-0de5-47f4-b6e9-b34709ee1304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:30.328942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1056/2087] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:30.333436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9ec9-2751-4e57-856c-f00c4bfe7b26 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.373194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:30.389219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b110e-0de5-47f4-b6e9-b34709ee1304 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.391698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2a0b110e-0de5-47f4-b6e9-b34709ee1304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:30.405061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9ec9-2751-4e57-856c-f00c4bfe7b26 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.410349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25e367-30c4-4e67-80c7-3055cb92669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.411010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25e367-30c4-4e67-80c7-3055cb92669a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.411291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25e367-30c4-4e67-80c7-3055cb92669a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.411563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25e367-30c4-4e67-80c7-3055cb92669a admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.412326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25e367-30c4-4e67-80c7-3055cb92669a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.414351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25e367-30c4-4e67-80c7-3055cb92669a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.418471 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b110e-0de5-47f4-b6e9-b34709ee1304 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.423780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.425649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.425649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.426277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] PATH_INFO: `/v3/projects/b24fb71f54464b39b35f4b3ff92736d8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.427475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b24fb71f54464b39b35f4b3ff92736d8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.429425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.533785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1032/2088] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:30 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:30.534852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25e367-30c4-4e67-80c7-3055cb92669a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.569916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b24fb71f54464b39b35f4b3ff92736d8', 'request_id': 'req-b8eb425a-eb19-489b-aa2e-341d7e2486c6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.575915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b24fb71f54464b39b35f4b3ff92736d8', 'request_id': 'req-b8eb425a-eb19-489b-aa2e-341d7e2486c6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.581551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b24fb71f54464b39b35f4b3ff92736d8', 'request_id': 'req-b8eb425a-eb19-489b-aa2e-341d7e2486c6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.598270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25e367-30c4-4e67-80c7-3055cb92669a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.653207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.653862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.654152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.654433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.655120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.657058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.700527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] The token cache is being invalidate because project b24fb71f54464b39b35f4b3ff92736d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:30.700816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.701478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.703052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1057/2089] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:30 2022] DELETE /identity/v3/projects/b24fb71f54464b39b35f4b3ff92736d8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:30.707516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.710589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:30.725150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb425a-eb19-489b-aa2e-341d7e2486c6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.729965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.733453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.733453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.733453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] PATH_INFO: `/v3/users/aaae9156704f4bab92251443f1ef4873` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.733453 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaae9156704f4bab92251443f1ef4873)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.734517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.758566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1033/2090] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:30 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:30.762543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.764142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:30.775905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a3996-301a-4aac-84cb-6d0ebe85b86b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.780633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424cfe2-924d-4044-8a63-45ac18ea5517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.786626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424cfe2-924d-4044-8a63-45ac18ea5517 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.786626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424cfe2-924d-4044-8a63-45ac18ea5517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.786626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424cfe2-924d-4044-8a63-45ac18ea5517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.786626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424cfe2-924d-4044-8a63-45ac18ea5517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.786626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424cfe2-924d-4044-8a63-45ac18ea5517 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.798801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1034/2091] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:30.800910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424cfe2-924d-4044-8a63-45ac18ea5517 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.813318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424cfe2-924d-4044-8a63-45ac18ea5517 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.818305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020683c-4b56-4b32-aa5c-f04dd1a630ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.818984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020683c-4b56-4b32-aa5c-f04dd1a630ca admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.819387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020683c-4b56-4b32-aa5c-f04dd1a630ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.819673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020683c-4b56-4b32-aa5c-f04dd1a630ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.820398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020683c-4b56-4b32-aa5c-f04dd1a630ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.822141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020683c-4b56-4b32-aa5c-f04dd1a630ca admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.833293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1035/2092] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:30.837518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020683c-4b56-4b32-aa5c-f04dd1a630ca admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:30.848170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020683c-4b56-4b32-aa5c-f04dd1a630ca admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:30.852841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef80a5-7101-446c-9b52-e4aefa91c523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:30.853843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef80a5-7101-446c-9b52-e4aefa91c523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.854126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef80a5-7101-446c-9b52-e4aefa91c523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.854399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef80a5-7101-446c-9b52-e4aefa91c523 admin admin] PATH_INFO: `/v3/projects/a0dcd540394b46a1afe283eac77c7534/users/810c0b3c853a467b99bd8050dae66f32/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.870206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef80a5-7101-446c-9b52-e4aefa91c523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0dcd540394b46a1afe283eac77c7534, user_id=810c0b3c853a467b99bd8050dae66f32, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:30.872880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef80a5-7101-446c-9b52-e4aefa91c523 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:30.961980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1036/2093] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:30 2022] PUT /identity/v3/projects/a0dcd540394b46a1afe283eac77c7534/users/810c0b3c853a467b99bd8050dae66f32/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:30.965177 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a791d-306c-4dae-874c-d16541f8750e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:30.965471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a791d-306c-4dae-874c-d16541f8750e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:30.965784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a791d-306c-4dae-874c-d16541f8750e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:30.968765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaae9156704f4bab92251443f1ef4873'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.977493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaae9156704f4bab92251443f1ef4873'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.989783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaae9156704f4bab92251443f1ef4873'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.990502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaae9156704f4bab92251443f1ef4873'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:30.993647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:31.001505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-660a791d-306c-4dae-874c-d16541f8750e None None] MFA Rules not processed for user `810c0b3c853a467b99bd8050dae66f32`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:31.002826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-660a791d-306c-4dae-874c-d16541f8750e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:31.027587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:31.065223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1037/2094] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:31.066363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaae9156704f4bab92251443f1ef4873'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:31.069589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a791d-306c-4dae-874c-d16541f8750e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.073235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1058/2095] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:30 2022] DELETE /identity/v3/users/aaae9156704f4bab92251443f1ef4873 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:31.140418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a791d-306c-4dae-874c-d16541f8750e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.198351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.199014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.199301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.199584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.200277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.202258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.293901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1038/2096] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:31 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:31.303167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.304640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:31.318622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928357f4-7b83-4063-9968-cf9d7fb69711 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.375575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacb70f-4a92-456f-9cc2-7102fae1d5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.376502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacb70f-4a92-456f-9cc2-7102fae1d5fe admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.376789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacb70f-4a92-456f-9cc2-7102fae1d5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.377240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacb70f-4a92-456f-9cc2-7102fae1d5fe admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.377939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacb70f-4a92-456f-9cc2-7102fae1d5fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.380411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacb70f-4a92-456f-9cc2-7102fae1d5fe admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.474122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.476281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:31.489439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820eb050-d5a9-4b5f-b4ac-3d3197520db4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.494341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.495208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.495550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.495834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] PATH_INFO: `/v3/projects/5c6eda9adb3445c7beba0f0c4e5a740b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.497019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6eda9adb3445c7beba0f0c4e5a740b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.499068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.558287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1059/2097] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:31 2022] POST /identity/v3/users => generated 300 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:31.562099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacb70f-4a92-456f-9cc2-7102fae1d5fe admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.572177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacb70f-4a92-456f-9cc2-7102fae1d5fe admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.576685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025436-ed4a-4be5-902b-43ad496d8f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.577346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025436-ed4a-4be5-902b-43ad496d8f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.577630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025436-ed4a-4be5-902b-43ad496d8f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.577905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83025436-ed4a-4be5-902b-43ad496d8f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.578608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83025436-ed4a-4be5-902b-43ad496d8f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.580238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83025436-ed4a-4be5-902b-43ad496d8f35 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.590819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1060/2098] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:31.595021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025436-ed4a-4be5-902b-43ad496d8f35 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.604467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83025436-ed4a-4be5-902b-43ad496d8f35 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.609062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.609929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.610216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.610497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] PATH_INFO: `/v3/projects/1f6fd1702fd048679ac43dda5b568198/users/a14f7f3d5eab43838d065907a4ed01d2/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.612254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:31.625918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6fd1702fd048679ac43dda5b568198, user_id=a14f7f3d5eab43838d065907a4ed01d2, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.629200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.630244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6eda9adb3445c7beba0f0c4e5a740b', 'request_id': 'req-76deac3d-20a2-456a-97d7-b87bfd768615'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:31.637070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6eda9adb3445c7beba0f0c4e5a740b', 'request_id': 'req-76deac3d-20a2-456a-97d7-b87bfd768615'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:31.642737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6eda9adb3445c7beba0f0c4e5a740b', 'request_id': 'req-76deac3d-20a2-456a-97d7-b87bfd768615'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:31.711913 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] The token cache is being invalidate because project 5c6eda9adb3445c7beba0f0c4e5a740b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:31.712206 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:31.712858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-76deac3d-20a2-456a-97d7-b87bfd768615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:31.713292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1061/2099] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:31 2022] PUT /identity/v3/projects/1f6fd1702fd048679ac43dda5b568198/users/a14f7f3d5eab43838d065907a4ed01d2/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:31.716948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.717166 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1039/2100] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:31 2022] DELETE /identity/v3/projects/5c6eda9adb3445c7beba0f0c4e5a740b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:31.719809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:31.727149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c5e4-5cda-4b60-aa3a-a717b05f8bf4 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.727439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c5e4-5cda-4b60-aa3a-a717b05f8bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.727720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c5e4-5cda-4b60-aa3a-a717b05f8bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.746558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7b75c5e4-5cda-4b60-aa3a-a717b05f8bf4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:31.747850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b75c5e4-5cda-4b60-aa3a-a717b05f8bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:31.798918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264272b-920a-4d42-8434-aa0983a1687f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.806303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc44e7-e8a3-4186-a403-86509789b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.806971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc44e7-e8a3-4186-a403-86509789b7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.807264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc44e7-e8a3-4186-a403-86509789b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.807551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc44e7-e8a3-4186-a403-86509789b7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.808283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc44e7-e8a3-4186-a403-86509789b7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.809911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc44e7-e8a3-4186-a403-86509789b7cb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.817356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1040/2101] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:31.824834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1062/2102] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:31.825840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc44e7-e8a3-4186-a403-86509789b7cb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.826363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1590ab0-2b2b-46f9-bd80-40a44e7c4b51 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.826659 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1590ab0-2b2b-46f9-bd80-40a44e7c4b51 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.826944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1590ab0-2b2b-46f9-bd80-40a44e7c4b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.835867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc44e7-e8a3-4186-a403-86509789b7cb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.840631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e446e2e-7a03-42c5-afe5-79f9f033cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.841550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e446e2e-7a03-42c5-afe5-79f9f033cbc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.841880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e446e2e-7a03-42c5-afe5-79f9f033cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.842160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e446e2e-7a03-42c5-afe5-79f9f033cbc0 admin admin] PATH_INFO: `/v3/projects/1f6fd1702fd048679ac43dda5b568198/users/a14f7f3d5eab43838d065907a4ed01d2/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.844120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e446e2e-7a03-42c5-afe5-79f9f033cbc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6fd1702fd048679ac43dda5b568198, user_id=a14f7f3d5eab43838d065907a4ed01d2, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.845760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d1590ab0-2b2b-46f9-bd80-40a44e7c4b51 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:31.846838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e446e2e-7a03-42c5-afe5-79f9f033cbc0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.846991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d1590ab0-2b2b-46f9-bd80-40a44e7c4b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:31.856379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1041/2103] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:31.857726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1063/2104] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:31 2022] PUT /identity/v3/projects/1f6fd1702fd048679ac43dda5b568198/users/a14f7f3d5eab43838d065907a4ed01d2/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:31.869719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1590ab0-2b2b-46f9-bd80-40a44e7c4b51 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.877062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88dd37-e80a-4a94-a335-1362e2d79607 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.877693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88dd37-e80a-4a94-a335-1362e2d79607 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.877997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88dd37-e80a-4a94-a335-1362e2d79607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.880467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1590ab0-2b2b-46f9-bd80-40a44e7c4b51 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.909829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1c88dd37-e80a-4a94-a335-1362e2d79607 None None] MFA Rules not processed for user `a14f7f3d5eab43838d065907a4ed01d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:31.911051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1c88dd37-e80a-4a94-a335-1362e2d79607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:31.931234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:31.944359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.945044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.945370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.945737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.946460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:31.946768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.948591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.963680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1042/2105] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:37:31.964908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.974114 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c357dae-c3f2-41c1-ab5a-5b007501bd8e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:31.978645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112ae2-cf59-4842-a485-3ee8dedc934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:31.979275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112ae2-cf59-4842-a485-3ee8dedc934e admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:31.979560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112ae2-cf59-4842-a485-3ee8dedc934e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:31.979839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112ae2-cf59-4842-a485-3ee8dedc934e admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:31.980515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7112ae2-cf59-4842-a485-3ee8dedc934e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:31.982446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7112ae2-cf59-4842-a485-3ee8dedc934e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:31.989399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1064/2106] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:31.997278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88dd37-e80a-4a94-a335-1362e2d79607 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:31.998782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1c88dd37-e80a-4a94-a335-1362e2d79607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:32.040673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1043/2107] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:31 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:32.044986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112ae2-cf59-4842-a485-3ee8dedc934e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:32.059595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88dd37-e80a-4a94-a335-1362e2d79607 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:32.071356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:32.109655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112ae2-cf59-4842-a485-3ee8dedc934e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:32.110610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d2ad0-e1c5-40e0-986f-1dbde06be43b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:32.111296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d2ad0-e1c5-40e0-986f-1dbde06be43b service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:32.111580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d2ad0-e1c5-40e0-986f-1dbde06be43b service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:32.111876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d2ad0-e1c5-40e0-986f-1dbde06be43b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:32.115329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db853708-1801-4ddb-a519-b7eeb9b8fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:32.116000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db853708-1801-4ddb-a519-b7eeb9b8fa62 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:32.116366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db853708-1801-4ddb-a519-b7eeb9b8fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:32.116733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db853708-1801-4ddb-a519-b7eeb9b8fa62 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:32.117459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db853708-1801-4ddb-a519-b7eeb9b8fa62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:32.119382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db853708-1801-4ddb-a519-b7eeb9b8fa62 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:32.171058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d2ad0-e1c5-40e0-986f-1dbde06be43b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:32.172506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d2ad0-e1c5-40e0-986f-1dbde06be43b service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:32.182573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1065/2108] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:32.195475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1044/2109] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:32 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:32.196747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d2ad0-e1c5-40e0-986f-1dbde06be43b service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:32.207381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d2ad0-e1c5-40e0-986f-1dbde06be43b service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:32.212032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc087e-b5e5-4e98-9609-856059b8e08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:32.212682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc087e-b5e5-4e98-9609-856059b8e08e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:32.212975 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc087e-b5e5-4e98-9609-856059b8e08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:32.213292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc087e-b5e5-4e98-9609-856059b8e08e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:32.214051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc087e-b5e5-4e98-9609-856059b8e08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:32.215639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc087e-b5e5-4e98-9609-856059b8e08e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:32.231764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1066/2110] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:32.232622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db853708-1801-4ddb-a519-b7eeb9b8fa62 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:32.244009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db853708-1801-4ddb-a519-b7eeb9b8fa62 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:32.248753 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b9374-d542-413b-80f5-60e0c81103ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:32.249419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9374-d542-413b-80f5-60e0c81103ab admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:32.249700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9374-d542-413b-80f5-60e0c81103ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:32.249974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b9374-d542-413b-80f5-60e0c81103ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:32.250688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b9374-d542-413b-80f5-60e0c81103ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:32.252277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b9374-d542-413b-80f5-60e0c81103ab admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:32.262410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1045/2111] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:32.266292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc087e-b5e5-4e98-9609-856059b8e08e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:32.276356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc087e-b5e5-4e98-9609-856059b8e08e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:32.281256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:32.282159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:32.282448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:32.282726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] PATH_INFO: `/v3/projects/53678ca1bba445e2ba0ff350c5224f73/users/df0d107479cd408289886752b693e959/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:32.291789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:32.298328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53678ca1bba445e2ba0ff350c5224f73, user_id=df0d107479cd408289886752b693e959, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:32.301550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:32.403546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1067/2112] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:32 2022] PUT /identity/v3/projects/53678ca1bba445e2ba0ff350c5224f73/users/df0d107479cd408289886752b693e959/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:32.413924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99925992-c6e2-499d-ba4a-fda2f0113516 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:32.414222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99925992-c6e2-499d-ba4a-fda2f0113516 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:32.414509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99925992-c6e2-499d-ba4a-fda2f0113516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:32.440355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-99925992-c6e2-499d-ba4a-fda2f0113516 None None] MFA Rules not processed for user `df0d107479cd408289886752b693e959`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:32.442179 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-99925992-c6e2-499d-ba4a-fda2f0113516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:32.507617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1046/2113] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:32.532320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:32.534170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:32.594079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e570f-d732-4f6b-9475-b1b932c40957 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:32.597796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee1a6f-4edc-4df3-9c7c-03d86a70a8bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:32.598438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee1a6f-4edc-4df3-9c7c-03d86a70a8bd service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:32.598726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee1a6f-4edc-4df3-9c7c-03d86a70a8bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:32.599005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee1a6f-4edc-4df3-9c7c-03d86a70a8bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:32.607439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee1a6f-4edc-4df3-9c7c-03d86a70a8bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:32.608904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee1a6f-4edc-4df3-9c7c-03d86a70a8bd service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:32.618933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1068/2114] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:33.333619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99925992-c6e2-499d-ba4a-fda2f0113516 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:33.347807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99925992-c6e2-499d-ba4a-fda2f0113516 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:33.351454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:33.352099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:33.352383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:33.352662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:33.354656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:33.417981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:33.419495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:33.428991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1047/2115] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:34.452261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee1a6f-4edc-4df3-9c7c-03d86a70a8bd service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:34.460995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee1a6f-4edc-4df3-9c7c-03d86a70a8bd service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:34.464654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:34.465313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:34.465600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:34.465879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:34.467754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:34.528409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:34.529887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:34.539509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1069/2116] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:37.607446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:37.616487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e960b-3d98-48a6-82c7-19a3e1271cdd service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:37.620153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:37.620790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:37.621072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:37.621423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:37.623180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:37.649425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:37.685705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:37.687119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:37.696429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1048/2117] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:44.535309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=59,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:44.738279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:44.811895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:44.813979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:44.823155 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b973ba8-5b57-40ac-a16e-6081917af18a service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:44.828923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:44.829952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:44.830243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:44.830528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] PATH_INFO: `/v3/users/df0d107479cd408289886752b693e959` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:44.832035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0d107479cd408289886752b693e959)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:44.832545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d690c-2099-4d18-ba57-f165dfa74bfb service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:44.834477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:44.840458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:44.841734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:44.842340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:44.842821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] PATH_INFO: `/v3/users/e514c31512ba40a38e82d73c381b0246` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:44.846038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e514c31512ba40a38e82d73c381b0246)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:44.848678 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:45.063579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0d107479cd408289886752b693e959'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.063766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e514c31512ba40a38e82d73c381b0246'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.070561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0d107479cd408289886752b693e959'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.080240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e514c31512ba40a38e82d73c381b0246'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.081135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e514c31512ba40a38e82d73c381b0246'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.082984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e514c31512ba40a38e82d73c381b0246'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.083428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0d107479cd408289886752b693e959'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.084188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0d107479cd408289886752b693e959'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.186810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e514c31512ba40a38e82d73c381b0246'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.200503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1049/2118] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:44 2022] DELETE /identity/v3/users/e514c31512ba40a38e82d73c381b0246 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:45.202676 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:45.202776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:45.267237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a2a89c-a3a5-4723-b5bb-65b2899806c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0d107479cd408289886752b693e959'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:45.275776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1070/2119] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:44 2022] DELETE /identity/v3/users/df0d107479cd408289886752b693e959 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:45.277852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020c7dd-5cae-4ce6-8b65-130c56fae19f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:45.278230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020c7dd-5cae-4ce6-8b65-130c56fae19f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:45.278639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020c7dd-5cae-4ce6-8b65-130c56fae19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:45.288246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdd2af-a1f8-485a-ba81-a7e89dbc58fb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:45.297781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1020c7dd-5cae-4ce6-8b65-130c56fae19f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:45.299108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1020c7dd-5cae-4ce6-8b65-130c56fae19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:45.344344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:45.345061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:45.345352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:45.345591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:45.347543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:45.354281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:45.379669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1071/2120] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:45.391456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf9e4f-af91-4589-bac9-f473d2ff00b3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:45.391456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf9e4f-af91-4589-bac9-f473d2ff00b3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:45.391456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf9e4f-af91-4589-bac9-f473d2ff00b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:45.411412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:45.413050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:45.413655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c4bf9e4f-af91-4589-bac9-f473d2ff00b3 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:45.415060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c4bf9e4f-af91-4589-bac9-f473d2ff00b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:45.425245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1072/2121] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:45.427953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf9e4f-af91-4589-bac9-f473d2ff00b3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:45.429296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c4bf9e4f-af91-4589-bac9-f473d2ff00b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:45.435924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1050/2122] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:45.437123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:45.439298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:45.453479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:45.501038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252473d8-498c-4adc-a85b-5059e327ea33 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:45.504060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf9e4f-af91-4589-bac9-f473d2ff00b3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:45.505860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:45.506627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:45.506970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:45.507256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:45.509458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:45.509599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:45.509976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:45.510406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:45.518031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:45.519298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:45.520231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:45.522321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:45.531270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1051/2123] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:45.533747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1073/2124] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:46.021726 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:46.024386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:46.053969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca0f42-a9cd-4c98-9626-a117b8ce06e2 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:46.065472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:46.068629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:46.069899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:46.071528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.076033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.076560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] PATH_INFO: `/v3/projects/53678ca1bba445e2ba0ff350c5224f73` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.081493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] RBAC: Authorizing `identity:delete_project(project_id=53678ca1bba445e2ba0ff350c5224f73)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:46.084530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:46.095842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad036b2-d5c8-4c8d-b408-54aa42ad1f8f service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:46.102043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:46.103258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.103733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.104978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] PATH_INFO: `/v3/projects/8feafd08250a444da0fc8b51cfcd22c8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.107375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8feafd08250a444da0fc8b51cfcd22c8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:46.111374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:46.221696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53678ca1bba445e2ba0ff350c5224f73', 'request_id': 'req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.228538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53678ca1bba445e2ba0ff350c5224f73', 'request_id': 'req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.236686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53678ca1bba445e2ba0ff350c5224f73', 'request_id': 'req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.252826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8feafd08250a444da0fc8b51cfcd22c8', 'request_id': 'req-e0750dce-a9c6-4d55-8262-97bca9c271d2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.259463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8feafd08250a444da0fc8b51cfcd22c8', 'request_id': 'req-e0750dce-a9c6-4d55-8262-97bca9c271d2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.266493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8feafd08250a444da0fc8b51cfcd22c8', 'request_id': 'req-e0750dce-a9c6-4d55-8262-97bca9c271d2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.381841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] The token cache is being invalidate because project 53678ca1bba445e2ba0ff350c5224f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:46.382141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.382774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7d2ef63b-7941-45d2-9f2d-c37b2d16515e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.384331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1052/2125] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:46 2022] DELETE /identity/v3/projects/53678ca1bba445e2ba0ff350c5224f73 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:46.389000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d9601-3395-4511-b808-80291dd1d042 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.389324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d9601-3395-4511-b808-80291dd1d042 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.389675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d9601-3395-4511-b808-80291dd1d042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.419624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8d2d9601-3395-4511-b808-80291dd1d042 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:46.420816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8d2d9601-3395-4511-b808-80291dd1d042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:46.437936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:46.483439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1053/2126] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:46.484666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] The token cache is being invalidate because project 8feafd08250a444da0fc8b51cfcd22c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:46.484967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.491579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344aed-3a11-4131-9ae5-673f2e4a68b8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.492031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344aed-3a11-4131-9ae5-673f2e4a68b8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.492311 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344aed-3a11-4131-9ae5-673f2e4a68b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.497776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.499601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1074/2127] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:46 2022] DELETE /identity/v3/projects/8feafd08250a444da0fc8b51cfcd22c8 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:46.503794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:46.506306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:46.515348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-10344aed-3a11-4131-9ae5-673f2e4a68b8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:46.516671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-10344aed-3a11-4131-9ae5-673f2e4a68b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:46.520925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0750dce-a9c6-4d55-8262-97bca9c271d2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:46.526435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:46.526981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1054/2128] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:46.527508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.527793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.528068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] PATH_INFO: `/v3/users/04a58755751845028093f563d636ee57` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.529279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a58755751845028093f563d636ee57)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:46.530652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10344aed-3a11-4131-9ae5-673f2e4a68b8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:46.532137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:46.540941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10344aed-3a11-4131-9ae5-673f2e4a68b8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:46.546693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:46.547009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.547366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.547722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.548786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:46.548904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:46.550929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:46.566861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1055/2129] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:46.571016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:46.580624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8101537-4261-4547-af71-5f70487b1785 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:46.585363 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287cf42-657b-4f56-82d3-85d6649cc66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:46.586014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287cf42-657b-4f56-82d3-85d6649cc66b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.586295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287cf42-657b-4f56-82d3-85d6649cc66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.586573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287cf42-657b-4f56-82d3-85d6649cc66b admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.587256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287cf42-657b-4f56-82d3-85d6649cc66b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:46.589180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287cf42-657b-4f56-82d3-85d6649cc66b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:46.702059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1056/2130] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:46 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:46.706170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287cf42-657b-4f56-82d3-85d6649cc66b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:46.754123 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a58755751845028093f563d636ee57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.761984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a58755751845028093f563d636ee57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.772192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287cf42-657b-4f56-82d3-85d6649cc66b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:46.775825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a58755751845028093f563d636ee57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.776610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a58755751845028093f563d636ee57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.781541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40f3bb-eb9e-4b6c-894d-e4e03f433965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:46.782224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40f3bb-eb9e-4b6c-894d-e4e03f433965 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.782511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40f3bb-eb9e-4b6c-894d-e4e03f433965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.782792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40f3bb-eb9e-4b6c-894d-e4e03f433965 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.783490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40f3bb-eb9e-4b6c-894d-e4e03f433965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:46.786044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40f3bb-eb9e-4b6c-894d-e4e03f433965 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:46.819925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a58755751845028093f563d636ee57'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:46.826589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1075/2131] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:46 2022] DELETE /identity/v3/users/04a58755751845028093f563d636ee57 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:46.890418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1057/2132] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:46 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:37:46.895408 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:46.897318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:46.925243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:46.979061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c7aa5-9375-4527-823c-9724de7cdb84 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:46.983965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8196747-b55b-46c8-b864-6f35759cc314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:46.984727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8196747-b55b-46c8-b864-6f35759cc314 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:46.985244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8196747-b55b-46c8-b864-6f35759cc314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:46.985566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8196747-b55b-46c8-b864-6f35759cc314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:46.986296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8196747-b55b-46c8-b864-6f35759cc314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:46.988245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8196747-b55b-46c8-b864-6f35759cc314 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:46.999221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1076/2133] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:47.003329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40f3bb-eb9e-4b6c-894d-e4e03f433965 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:47.013551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40f3bb-eb9e-4b6c-894d-e4e03f433965 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:47.022502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270c9d5-8cff-4050-a0fc-00149efae838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:47.027315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270c9d5-8cff-4050-a0fc-00149efae838 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:47.027315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270c9d5-8cff-4050-a0fc-00149efae838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:47.027315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270c9d5-8cff-4050-a0fc-00149efae838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:47.027315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270c9d5-8cff-4050-a0fc-00149efae838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:47.028961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270c9d5-8cff-4050-a0fc-00149efae838 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:47.046098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1058/2134] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:47.050104 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8196747-b55b-46c8-b864-6f35759cc314 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:47.060781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8196747-b55b-46c8-b864-6f35759cc314 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:47.065735 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:47.067357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:47.067462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:47.067705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] PATH_INFO: `/v3/projects/1c02e4c8a1d449f087a42405a3105f08/users/041649a172914799aaf4785fc1d4f242/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:47.089335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c02e4c8a1d449f087a42405a3105f08, user_id=041649a172914799aaf4785fc1d4f242, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:47.094842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:47.148273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1077/2135] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:47 2022] PUT /identity/v3/projects/1c02e4c8a1d449f087a42405a3105f08/users/041649a172914799aaf4785fc1d4f242/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:47.159107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ebd-192f-4a0c-965a-6e7ac975dfd7 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:47.159609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ebd-192f-4a0c-965a-6e7ac975dfd7 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:47.160041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ebd-192f-4a0c-965a-6e7ac975dfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:47.193230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3b725ebd-192f-4a0c-965a-6e7ac975dfd7 None None] MFA Rules not processed for user `041649a172914799aaf4785fc1d4f242`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:47.194449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3b725ebd-192f-4a0c-965a-6e7ac975dfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:47.260529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1059/2136] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:47.320111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:47.322559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:47.392452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d68243-c276-4e3d-94b8-ed32a45635e5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:47.398501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:47.399366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:47.399650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:47.399930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] PATH_INFO: `/v3/projects/56322bc8c6a14de498f19d610dc8e1ef` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:47.401136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=56322bc8c6a14de498f19d610dc8e1ef)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:47.403090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:47.460631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725ebd-192f-4a0c-965a-6e7ac975dfd7 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:47.462208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3b725ebd-192f-4a0c-965a-6e7ac975dfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:47.528413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725ebd-192f-4a0c-965a-6e7ac975dfd7 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:47.545401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56322bc8c6a14de498f19d610dc8e1ef', 'request_id': 'req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.553451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56322bc8c6a14de498f19d610dc8e1ef', 'request_id': 'req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.558685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56322bc8c6a14de498f19d610dc8e1ef', 'request_id': 'req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.609998 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba61f2-1088-4bc6-8259-d575f69be497 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:47.610675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba61f2-1088-4bc6-8259-d575f69be497 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:47.610958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba61f2-1088-4bc6-8259-d575f69be497 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:47.611235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba61f2-1088-4bc6-8259-d575f69be497 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:47.626967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] The token cache is being invalidate because project 56322bc8c6a14de498f19d610dc8e1ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:47.627267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.627704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:47.627956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.629979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1078/2137] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:47 2022] DELETE /identity/v3/projects/56322bc8c6a14de498f19d610dc8e1ef => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:47.633871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:47.635707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:47.715456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba61f2-1088-4bc6-8259-d575f69be497 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:47.717515 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba61f2-1088-4bc6-8259-d575f69be497 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:47.740996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1060/2138] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:47.758564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a42d4a-9ccc-479c-ae96-9aea6bf0dadd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:47.763530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:47.764405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:47.764691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:47.764972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] PATH_INFO: `/v3/users/9734245a099444c098cf25bececdd637` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:47.766347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9734245a099444c098cf25bececdd637)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:47.768280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:47.778680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba61f2-1088-4bc6-8259-d575f69be497 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:47.791701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba61f2-1088-4bc6-8259-d575f69be497 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:47.800595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:47.802214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:47.802493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:47.802772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] PATH_INFO: `/v3/users/041649a172914799aaf4785fc1d4f242` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:47.804054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=041649a172914799aaf4785fc1d4f242)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:47.806112 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:47.932477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9734245a099444c098cf25bececdd637'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.935967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:47.942031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9734245a099444c098cf25bececdd637'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.960866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9734245a099444c098cf25bececdd637'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.965135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9734245a099444c098cf25bececdd637'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:47.989726 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041649a172914799aaf4785fc1d4f242'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.001014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041649a172914799aaf4785fc1d4f242'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.001700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041649a172914799aaf4785fc1d4f242'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.002332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041649a172914799aaf4785fc1d4f242'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.020809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b805ab45-6883-4be9-8b5f-64fc6b778cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9734245a099444c098cf25bececdd637'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.027206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1079/2139] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:47 2022] DELETE /identity/v3/users/9734245a099444c098cf25bececdd637 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:48.101217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041649a172914799aaf4785fc1d4f242'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.124814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1061/2140] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:47 2022] DELETE /identity/v3/users/041649a172914799aaf4785fc1d4f242 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:48.126567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2478c-b393-44c6-b443-9a7945f9d51b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:48.127056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2478c-b393-44c6-b443-9a7945f9d51b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:48.127148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2478c-b393-44c6-b443-9a7945f9d51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:48.145881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6aa2478c-b393-44c6-b443-9a7945f9d51b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:48.147109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6aa2478c-b393-44c6-b443-9a7945f9d51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:48.205567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1080/2141] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:48.215455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:48.217283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:48.276694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ed323-d2b1-4d3c-9ef5-1b5c3e2d56f3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:48.280368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3f8ea-60ec-498a-8ed7-e8cecbd01251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:48.281020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3f8ea-60ec-498a-8ed7-e8cecbd01251 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:48.281346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3f8ea-60ec-498a-8ed7-e8cecbd01251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:48.281624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3f8ea-60ec-498a-8ed7-e8cecbd01251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:48.290904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3f8ea-60ec-498a-8ed7-e8cecbd01251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:48.292351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3f8ea-60ec-498a-8ed7-e8cecbd01251 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:48.302765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1062/2142] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:48.427581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:48.543509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2478c-b393-44c6-b443-9a7945f9d51b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:48.544983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6aa2478c-b393-44c6-b443-9a7945f9d51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:48.562164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2478c-b393-44c6-b443-9a7945f9d51b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:48.562436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:48.563076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:48.563366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:48.563648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] PATH_INFO: `/v3/projects/8f65e17dbea549b29dcd9e9263c54151` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:48.564880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f65e17dbea549b29dcd9e9263c54151)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:48.566859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:48.698333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f65e17dbea549b29dcd9e9263c54151', 'request_id': 'req-8a337754-934e-4209-8d7c-3db225a688ce'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.704878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f65e17dbea549b29dcd9e9263c54151', 'request_id': 'req-8a337754-934e-4209-8d7c-3db225a688ce'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.711229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f65e17dbea549b29dcd9e9263c54151', 'request_id': 'req-8a337754-934e-4209-8d7c-3db225a688ce'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.806003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] The token cache is being invalidate because project 8f65e17dbea549b29dcd9e9263c54151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:48.806941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.807672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8a337754-934e-4209-8d7c-3db225a688ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:48.814898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1081/2143] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:48 2022] DELETE /identity/v3/projects/8f65e17dbea549b29dcd9e9263c54151 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:48.815297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2402d3-9785-4977-86e9-c046edb996d8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:48.820064 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2402d3-9785-4977-86e9-c046edb996d8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:48.820064 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2402d3-9785-4977-86e9-c046edb996d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:48.835932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8f2402d3-9785-4977-86e9-c046edb996d8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:48.837207 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8f2402d3-9785-4977-86e9-c046edb996d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:48.924058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1063/2144] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:48.931979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aa509-9445-49f8-8360-0e4dd5789f5f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:48.932294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aa509-9445-49f8-8360-0e4dd5789f5f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:48.932581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aa509-9445-49f8-8360-0e4dd5789f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:48.957988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2402d3-9785-4977-86e9-c046edb996d8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:48.963469 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8f2402d3-9785-4977-86e9-c046edb996d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:48.963756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-441aa509-9445-49f8-8360-0e4dd5789f5f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:48.964716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-441aa509-9445-49f8-8360-0e4dd5789f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:48.973856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1082/2145] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:48.974031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2402d3-9785-4977-86e9-c046edb996d8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:48.977726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441aa509-9445-49f8-8360-0e4dd5789f5f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:48.980970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:48.982026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:48.982310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:48.982587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] PATH_INFO: `/v3/projects/1c02e4c8a1d449f087a42405a3105f08` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:48.988224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441aa509-9445-49f8-8360-0e4dd5789f5f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:48.994288 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:48.999447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:48.999447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:48.999447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:48.999447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:48.999447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:48.999447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:49.012053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1083/2146] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:49.016003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c02e4c8a1d449f087a42405a3105f08)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:49.018231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:49.022305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:49.029995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b4cf8e-7477-4381-bdd0-fbd7b932157f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:49.036064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881faf8-25a1-4cde-b8a0-9e98994250d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:49.036742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881faf8-25a1-4cde-b8a0-9e98994250d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:49.037029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881faf8-25a1-4cde-b8a0-9e98994250d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:49.037358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881faf8-25a1-4cde-b8a0-9e98994250d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:49.038100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881faf8-25a1-4cde-b8a0-9e98994250d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:49.040680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881faf8-25a1-4cde-b8a0-9e98994250d8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:49.139713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1084/2147] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:49 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:49.143836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881faf8-25a1-4cde-b8a0-9e98994250d8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:49.151455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:49.170367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c02e4c8a1d449f087a42405a3105f08', 'request_id': 'req-da855733-6f7f-4bd1-8ee9-474c057d105c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:49.178212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c02e4c8a1d449f087a42405a3105f08', 'request_id': 'req-da855733-6f7f-4bd1-8ee9-474c057d105c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:49.184546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c02e4c8a1d449f087a42405a3105f08', 'request_id': 'req-da855733-6f7f-4bd1-8ee9-474c057d105c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:49.226697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881faf8-25a1-4cde-b8a0-9e98994250d8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:49.274076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] The token cache is being invalidate because project 1c02e4c8a1d449f087a42405a3105f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:49.274349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:49.275078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:49.276754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1064/2148] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:48 2022] DELETE /identity/v3/projects/1c02e4c8a1d449f087a42405a3105f08 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:49.285057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:49.296603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80fe91-6224-4d9d-8576-dbaecd8305f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:49.297656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80fe91-6224-4d9d-8576-dbaecd8305f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:49.297946 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80fe91-6224-4d9d-8576-dbaecd8305f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:49.298228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80fe91-6224-4d9d-8576-dbaecd8305f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:49.299697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80fe91-6224-4d9d-8576-dbaecd8305f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:49.302171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c80fe91-6224-4d9d-8576-dbaecd8305f3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:49.401368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1085/2149] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:49 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:49.402209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:49.403798 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:49.446421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da855733-6f7f-4bd1-8ee9-474c057d105c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:49.459529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556812c-63ac-4ae9-9f95-9819d63324f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:49.460255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556812c-63ac-4ae9-9f95-9819d63324f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:49.460640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556812c-63ac-4ae9-9f95-9819d63324f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:49.461067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556812c-63ac-4ae9-9f95-9819d63324f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:49.461860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556812c-63ac-4ae9-9f95-9819d63324f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:49.465446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556812c-63ac-4ae9-9f95-9819d63324f0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:49.484136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1065/2150] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:49.493545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80fe91-6224-4d9d-8576-dbaecd8305f3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:49.504664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c80fe91-6224-4d9d-8576-dbaecd8305f3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:49.511998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5e499-7d95-45f9-87bb-88250d76519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:49.512980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5e499-7d95-45f9-87bb-88250d76519d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:49.520497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5e499-7d95-45f9-87bb-88250d76519d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:49.520497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5e499-7d95-45f9-87bb-88250d76519d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:49.520497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5e499-7d95-45f9-87bb-88250d76519d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:49.520497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5e499-7d95-45f9-87bb-88250d76519d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:49.528455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1086/2151] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:49.533205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556812c-63ac-4ae9-9f95-9819d63324f0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:49.544972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556812c-63ac-4ae9-9f95-9819d63324f0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:49.550938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:49.552058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:49.552345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:49.553231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] PATH_INFO: `/v3/projects/ccf50ff938ce4fdb8d24871054e0015d/users/db802295be064253ae59efcbfd12be8f/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:49.571566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf50ff938ce4fdb8d24871054e0015d, user_id=db802295be064253ae59efcbfd12be8f, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:49.574514 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:49.692487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1066/2152] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:49 2022] PUT /identity/v3/projects/ccf50ff938ce4fdb8d24871054e0015d/users/db802295be064253ae59efcbfd12be8f/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:49.697937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a71a5-8262-49d9-81df-fa08f29c1762 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:49.698232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a71a5-8262-49d9-81df-fa08f29c1762 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:49.698513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a71a5-8262-49d9-81df-fa08f29c1762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:49.731549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c22a71a5-8262-49d9-81df-fa08f29c1762 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:49.733502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c22a71a5-8262-49d9-81df-fa08f29c1762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:49.822505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1087/2153] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:49.832819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:49.834602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:49.905824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39f798-585a-4ce0-b857-41f89d3c8a2f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:49.910230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4205e98-aea7-4ba4-9d8f-569ac01fef17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:49.912828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4205e98-aea7-4ba4-9d8f-569ac01fef17 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:49.913251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4205e98-aea7-4ba4-9d8f-569ac01fef17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:49.913633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4205e98-aea7-4ba4-9d8f-569ac01fef17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:49.924151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4205e98-aea7-4ba4-9d8f-569ac01fef17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:49.925646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4205e98-aea7-4ba4-9d8f-569ac01fef17 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:49.936125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1067/2154] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:50.604532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a71a5-8262-49d9-81df-fa08f29c1762 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:50.606238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c22a71a5-8262-49d9-81df-fa08f29c1762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:50.618343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a71a5-8262-49d9-81df-fa08f29c1762 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:50.625583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:50.630247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:50.630247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:50.630247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] PATH_INFO: `/v3/users/9e4a3b3acd8345388faf58d236040872` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:50.630247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4a3b3acd8345388faf58d236040872)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:50.631342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:50.836378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4a3b3acd8345388faf58d236040872'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:50.843406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4a3b3acd8345388faf58d236040872'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:50.857314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4a3b3acd8345388faf58d236040872'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:50.859170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4a3b3acd8345388faf58d236040872'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:50.913051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:50.969432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4a3b3acd8345388faf58d236040872'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:50.975933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1088/2155] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:50 2022] DELETE /identity/v3/users/9e4a3b3acd8345388faf58d236040872 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:50.980420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88378db-c5ae-43bc-95c8-57f31924fd08 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:50.980870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88378db-c5ae-43bc-95c8-57f31924fd08 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:50.981131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88378db-c5ae-43bc-95c8-57f31924fd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:50.999775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e88378db-c5ae-43bc-95c8-57f31924fd08 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:51.000991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e88378db-c5ae-43bc-95c8-57f31924fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.060622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1068/2156] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:51.072116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:51.073828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.133727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9aabf-3d5e-432c-bdc2-a726645302a6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:51.137370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d14a-8d68-4cc5-ad1d-cb95e57c5ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:51.138141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d14a-8d68-4cc5-ad1d-cb95e57c5ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:51.138481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d14a-8d68-4cc5-ad1d-cb95e57c5ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:51.138758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d14a-8d68-4cc5-ad1d-cb95e57c5ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:51.149018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1d14a-8d68-4cc5-ad1d-cb95e57c5ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:51.150484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1d14a-8d68-4cc5-ad1d-cb95e57c5ee3 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:51.160313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1089/2157] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:51.318462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88378db-c5ae-43bc-95c8-57f31924fd08 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:51.319897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e88378db-c5ae-43bc-95c8-57f31924fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.332506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88378db-c5ae-43bc-95c8-57f31924fd08 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:51.337472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:51.338435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:51.339277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:51.339623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] PATH_INFO: `/v3/users/e6e9cfec6bec43b6a17aba51cc0f991d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:51.340884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e9cfec6bec43b6a17aba51cc0f991d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:51.342927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:51.411396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4535f-dd0d-4d59-ad29-3e5cac849046 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:51.411607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4535f-dd0d-4d59-ad29-3e5cac849046 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:51.412068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4535f-dd0d-4d59-ad29-3e5cac849046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:51.447488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-48e4535f-dd0d-4d59-ad29-3e5cac849046 None None] MFA Rules not processed for user `db802295be064253ae59efcbfd12be8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:51.449425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-48e4535f-dd0d-4d59-ad29-3e5cac849046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.567400 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1090/2158] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:51.580343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4535f-dd0d-4d59-ad29-3e5cac849046 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:51.582142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-48e4535f-dd0d-4d59-ad29-3e5cac849046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.609424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e9cfec6bec43b6a17aba51cc0f991d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:51.623465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:51.625983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e9cfec6bec43b6a17aba51cc0f991d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:51.626804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e9cfec6bec43b6a17aba51cc0f991d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:51.627492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e9cfec6bec43b6a17aba51cc0f991d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:51.654415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4535f-dd0d-4d59-ad29-3e5cac849046 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:51.658083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:51.658725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:51.659016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:51.659337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:51.661205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.671926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:51.673373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:51.682303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1091/2159] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:51.718706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c13db66c-242b-4c08-bc68-aba87f30bba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e9cfec6bec43b6a17aba51cc0f991d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:51.729434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:51.731319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.737005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1069/2160] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:51 2022] DELETE /identity/v3/users/e6e9cfec6bec43b6a17aba51cc0f991d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:51.739008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413c82-54bc-4f0f-886a-6f0aebe9b2c2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:51.739468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413c82-54bc-4f0f-886a-6f0aebe9b2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:51.739877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413c82-54bc-4f0f-886a-6f0aebe9b2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:51.753834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82deb3a4-3c78-49c8-9762-b2be0d82b4f7 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:51.763996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6a413c82-54bc-4f0f-886a-6f0aebe9b2c2 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:51.766094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6a413c82-54bc-4f0f-886a-6f0aebe9b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.794662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:51.850085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1070/2161] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:51.855090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:51.855966 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:51.856247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:51.856520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] PATH_INFO: `/v3/projects/b8f3144aed5c4c64a2d723dfc09db682` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:51.867089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f3144aed5c4c64a2d723dfc09db682)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:51.867089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:51.867321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a413c82-54bc-4f0f-886a-6f0aebe9b2c2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:51.867321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6a413c82-54bc-4f0f-886a-6f0aebe9b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:51.918239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a413c82-54bc-4f0f-886a-6f0aebe9b2c2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:51.921904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bc2a2-8950-47aa-aca3-c8c121d8f185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:51.922551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bc2a2-8950-47aa-aca3-c8c121d8f185 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:51.922830 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bc2a2-8950-47aa-aca3-c8c121d8f185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:51.923104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8bc2a2-8950-47aa-aca3-c8c121d8f185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:51.933498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bc2a2-8950-47aa-aca3-c8c121d8f185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:51.934939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8bc2a2-8950-47aa-aca3-c8c121d8f185 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:51.944293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1071/2162] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:51.991648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f3144aed5c4c64a2d723dfc09db682', 'request_id': 'req-9031307a-82e3-4ef3-b4b7-d2970cc606e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.001745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f3144aed5c4c64a2d723dfc09db682', 'request_id': 'req-9031307a-82e3-4ef3-b4b7-d2970cc606e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.008427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f3144aed5c4c64a2d723dfc09db682', 'request_id': 'req-9031307a-82e3-4ef3-b4b7-d2970cc606e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.077391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] The token cache is being invalidate because project b8f3144aed5c4c64a2d723dfc09db682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:52.077683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.078335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.080617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1092/2163] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:51 2022] DELETE /identity/v3/projects/b8f3144aed5c4c64a2d723dfc09db682 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:52.085581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bc2a2-8950-47aa-aca3-c8c121d8f185 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:52.100840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8bc2a2-8950-47aa-aca3-c8c121d8f185 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:52.163721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:52.164628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:52.164914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:52.165232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] PATH_INFO: `/v3/users/810c0b3c853a467b99bd8050dae66f32` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:52.166466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=810c0b3c853a467b99bd8050dae66f32)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:52.168429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:52.368531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:52.370690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:52.382697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '810c0b3c853a467b99bd8050dae66f32'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.393882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031307a-82e3-4ef3-b4b7-d2970cc606e1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:52.404781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:52.406173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:52.407251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:52.408069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:52.409261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] PATH_INFO: `/v3/projects/e50dec77ce95482684238cccd67838d8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:52.420550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50dec77ce95482684238cccd67838d8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:52.425432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:52.425529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '810c0b3c853a467b99bd8050dae66f32'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.426882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '810c0b3c853a467b99bd8050dae66f32'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.427725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '810c0b3c853a467b99bd8050dae66f32'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.536061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bb597de-7de9-4dd8-97df-0703e236c2bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '810c0b3c853a467b99bd8050dae66f32'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.536521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:52.550985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1072/2164] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:52 2022] DELETE /identity/v3/users/810c0b3c853a467b99bd8050dae66f32 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:52.565655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50dec77ce95482684238cccd67838d8', 'request_id': 'req-02b1412e-359f-458b-bf69-f5611a23d810'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.572093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50dec77ce95482684238cccd67838d8', 'request_id': 'req-02b1412e-359f-458b-bf69-f5611a23d810'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.577776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50dec77ce95482684238cccd67838d8', 'request_id': 'req-02b1412e-359f-458b-bf69-f5611a23d810'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.689261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] The token cache is being invalidate because project e50dec77ce95482684238cccd67838d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:52.689931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.690474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02b1412e-359f-458b-bf69-f5611a23d810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:52.694241 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1093/2165] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:52 2022] DELETE /identity/v3/projects/e50dec77ce95482684238cccd67838d8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:52.698153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc41923-7366-4f43-8f55-0a010b12e733 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:52.698449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc41923-7366-4f43-8f55-0a010b12e733 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:52.698784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc41923-7366-4f43-8f55-0a010b12e733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:52.718511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3cc41923-7366-4f43-8f55-0a010b12e733 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:52.719260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3cc41923-7366-4f43-8f55-0a010b12e733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:52.781901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1073/2166] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:52.790673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28244-bb1e-4242-914b-d933939d5cd6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:52.791160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28244-bb1e-4242-914b-d933939d5cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:52.791454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28244-bb1e-4242-914b-d933939d5cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:52.811384 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9af28244-bb1e-4242-914b-d933939d5cd6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:52.812636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9af28244-bb1e-4242-914b-d933939d5cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:52.820772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1094/2167] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:52.825400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc41923-7366-4f43-8f55-0a010b12e733 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:52.836292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc41923-7366-4f43-8f55-0a010b12e733 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:52.842253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:52.843349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:52.843719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:52.844094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:52.844925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:52.845357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:52.847102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:52.863825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1074/2168] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:37:52.867806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af28244-bb1e-4242-914b-d933939d5cd6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:52.877630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af28244-bb1e-4242-914b-d933939d5cd6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:52.882433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:52.883316 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:52.883754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:52.884022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:52.884869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:52.886781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:52.972057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1095/2169] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:52 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:52.976753 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:52.989941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72c0c9-0a46-491e-a84a-9a5c0b6aa692 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:53.071170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b879df-7ae9-431b-8fa9-1d5eaa3927cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:53.072172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b879df-7ae9-431b-8fa9-1d5eaa3927cb admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.072460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b879df-7ae9-431b-8fa9-1d5eaa3927cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.072738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b879df-7ae9-431b-8fa9-1d5eaa3927cb admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.073481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b879df-7ae9-431b-8fa9-1d5eaa3927cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:53.075403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b879df-7ae9-431b-8fa9-1d5eaa3927cb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:53.246150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:53.246314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1075/2170] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:52 2022] POST /identity/v3/users => generated 292 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:53.246993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b879df-7ae9-431b-8fa9-1d5eaa3927cb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:53.248263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:53.262167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2a344-0990-4cec-b30b-a9c77d9d039b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:53.265547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b879df-7ae9-431b-8fa9-1d5eaa3927cb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:53.268394 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:53.269585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.270257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.270695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] PATH_INFO: `/v3/projects/a0dcd540394b46a1afe283eac77c7534` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.270955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb0e27-15b0-41ef-9e00-ced9202703ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:53.271729 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb0e27-15b0-41ef-9e00-ced9202703ff admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.272026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb0e27-15b0-41ef-9e00-ced9202703ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.272310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb0e27-15b0-41ef-9e00-ced9202703ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.272827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0dcd540394b46a1afe283eac77c7534)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:53.273253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb0e27-15b0-41ef-9e00-ced9202703ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:53.275482 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:53.275617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb0e27-15b0-41ef-9e00-ced9202703ff admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:53.287777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1076/2171] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:53.291746 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb0e27-15b0-41ef-9e00-ced9202703ff admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:53.303451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb0e27-15b0-41ef-9e00-ced9202703ff admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:53.308947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeec540-8cb1-4f6a-9744-b290cef0af54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:53.309718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeec540-8cb1-4f6a-9744-b290cef0af54 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.310004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeec540-8cb1-4f6a-9744-b290cef0af54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.310286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeec540-8cb1-4f6a-9744-b290cef0af54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.311014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeec540-8cb1-4f6a-9744-b290cef0af54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:53.312619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeec540-8cb1-4f6a-9744-b290cef0af54 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:53.326801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1077/2172] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:53.328973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeec540-8cb1-4f6a-9744-b290cef0af54 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:53.342026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeec540-8cb1-4f6a-9744-b290cef0af54 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:53.348897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db0e51-bbb2-4b2c-b119-3674f122ea63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:53.350789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db0e51-bbb2-4b2c-b119-3674f122ea63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.351038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db0e51-bbb2-4b2c-b119-3674f122ea63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.351326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db0e51-bbb2-4b2c-b119-3674f122ea63 admin admin] PATH_INFO: `/v3/projects/b3f7c1cdb98740d8a855fb84f66ce550/users/33edd0878dc145ec874957b454d92e1f/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.368814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db0e51-bbb2-4b2c-b119-3674f122ea63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f7c1cdb98740d8a855fb84f66ce550, user_id=33edd0878dc145ec874957b454d92e1f, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:53.371581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db0e51-bbb2-4b2c-b119-3674f122ea63 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:53.490747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1078/2173] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:53 2022] PUT /identity/v3/projects/b3f7c1cdb98740d8a855fb84f66ce550/users/33edd0878dc145ec874957b454d92e1f/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:53.492137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908ed9-9580-407a-811e-78f543ad0134 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.492434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908ed9-9580-407a-811e-78f543ad0134 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.492713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf908ed9-9580-407a-811e-78f543ad0134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.511552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-cf908ed9-9580-407a-811e-78f543ad0134 None None] MFA Rules not processed for user `db802295be064253ae59efcbfd12be8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:53.512757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cf908ed9-9580-407a-811e-78f543ad0134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:53.513373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0dcd540394b46a1afe283eac77c7534', 'request_id': 'req-72be1f1f-5e53-4472-863d-b59a83df6778'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:53.520537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0dcd540394b46a1afe283eac77c7534', 'request_id': 'req-72be1f1f-5e53-4472-863d-b59a83df6778'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:53.526629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0dcd540394b46a1afe283eac77c7534', 'request_id': 'req-72be1f1f-5e53-4472-863d-b59a83df6778'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:53.571371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1079/2174] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:53.572785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebcc88-55fa-4805-a8a0-20924a3ba5ec None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.573073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebcc88-55fa-4805-a8a0-20924a3ba5ec None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.573408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebcc88-55fa-4805-a8a0-20924a3ba5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.602176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-13ebcc88-55fa-4805-a8a0-20924a3ba5ec None None] MFA Rules not processed for user `33edd0878dc145ec874957b454d92e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:53.603832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-13ebcc88-55fa-4805-a8a0-20924a3ba5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:53.636022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] The token cache is being invalidate because project a0dcd540394b46a1afe283eac77c7534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:53.636279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:53.638099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:53.642040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1096/2175] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:53 2022] DELETE /identity/v3/projects/a0dcd540394b46a1afe283eac77c7534 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:53.646600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:53.648632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:53.680442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1080/2176] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:53.692397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:53.693929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebcc88-55fa-4805-a8a0-20924a3ba5ec None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:53.695697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-13ebcc88-55fa-4805-a8a0-20924a3ba5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:53.704683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:53.772935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1f1f-5e53-4472-863d-b59a83df6778 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:53.775461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:53.776415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.777215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.777215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] PATH_INFO: `/v3/users/a14f7f3d5eab43838d065907a4ed01d2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.778570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a14f7f3d5eab43838d065907a4ed01d2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:53.780799 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebcc88-55fa-4805-a8a0-20924a3ba5ec None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:53.781290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:53.786118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d92b8f-ac0e-4fe2-b978-0d93d6e94e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:53.787449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d92b8f-ac0e-4fe2-b978-0d93d6e94e05 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:53.787837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d92b8f-ac0e-4fe2-b978-0d93d6e94e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:53.788125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d92b8f-ac0e-4fe2-b978-0d93d6e94e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:53.802880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d92b8f-ac0e-4fe2-b978-0d93d6e94e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:53.805269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d92b8f-ac0e-4fe2-b978-0d93d6e94e05 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:53.831593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1081/2177] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:54.000638 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a14f7f3d5eab43838d065907a4ed01d2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:54.009016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a14f7f3d5eab43838d065907a4ed01d2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:54.021329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a14f7f3d5eab43838d065907a4ed01d2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:54.022331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a14f7f3d5eab43838d065907a4ed01d2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:54.123991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-601694e7-b6fc-4369-9eb8-4f80e0e34935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a14f7f3d5eab43838d065907a4ed01d2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:54.131062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1097/2178] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:53 2022] DELETE /identity/v3/users/a14f7f3d5eab43838d065907a4ed01d2 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:54.639663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d92b8f-ac0e-4fe2-b978-0d93d6e94e05 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:54.655993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d92b8f-ac0e-4fe2-b978-0d93d6e94e05 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:54.662349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:54.729004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:54.730043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:54.730380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:54.730672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] PATH_INFO: `/v3/projects/1f6fd1702fd048679ac43dda5b568198` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:54.732475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f6fd1702fd048679ac43dda5b568198)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:54.734708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:54.908359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f6fd1702fd048679ac43dda5b568198', 'request_id': 'req-d67abf89-d84e-4076-9146-63be5460bb67'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:54.916213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f6fd1702fd048679ac43dda5b568198', 'request_id': 'req-d67abf89-d84e-4076-9146-63be5460bb67'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:54.922882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f6fd1702fd048679ac43dda5b568198', 'request_id': 'req-d67abf89-d84e-4076-9146-63be5460bb67'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:55.067323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] The token cache is being invalidate because project 1f6fd1702fd048679ac43dda5b568198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:55.068058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:55.068985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d67abf89-d84e-4076-9146-63be5460bb67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:55.071045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1082/2179] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:54 2022] DELETE /identity/v3/projects/1f6fd1702fd048679ac43dda5b568198 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:55.076703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ad5d-0128-483e-a0c0-8623c6033879 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.077552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ad5d-0128-483e-a0c0-8623c6033879 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.077841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ad5d-0128-483e-a0c0-8623c6033879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.104358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0fc7ad5d-0128-483e-a0c0-8623c6033879 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:55.104358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0fc7ad5d-0128-483e-a0c0-8623c6033879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:55.181231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1098/2180] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:55.183560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911e84-8734-4489-b712-e4e0d7883ce1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.183854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911e84-8734-4489-b712-e4e0d7883ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.184134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911e84-8734-4489-b712-e4e0d7883ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.205231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-07911e84-8734-4489-b712-e4e0d7883ce1 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:55.206887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-07911e84-8734-4489-b712-e4e0d7883ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:55.226490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1083/2181] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:55.231006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7ad5d-0128-483e-a0c0-8623c6033879 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:55.245397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7ad5d-0128-483e-a0c0-8623c6033879 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:55.250455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:55.251127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.253603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.253603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.253603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:55.253603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:55.254437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:55.266965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1099/2182] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:37:55.271966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911e84-8734-4489-b712-e4e0d7883ce1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:55.281476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911e84-8734-4489-b712-e4e0d7883ce1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:55.286852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ba2b5-21f6-490c-86ff-c67fd8d216e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:55.287553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ba2b5-21f6-490c-86ff-c67fd8d216e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.287836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ba2b5-21f6-490c-86ff-c67fd8d216e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.288142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ba2b5-21f6-490c-86ff-c67fd8d216e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.288846 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ba2b5-21f6-490c-86ff-c67fd8d216e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:55.290805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ba2b5-21f6-490c-86ff-c67fd8d216e4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:55.394299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1084/2183] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:55 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:55.399202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:55.409300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a2cab-62a9-4ddc-af12-39b5bbdd806e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:55.428902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:55.474658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc2ac4-3761-478c-9f98-1bf95511b62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:55.475362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc2ac4-3761-478c-9f98-1bf95511b62e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.475647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc2ac4-3761-478c-9f98-1bf95511b62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.475925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc2ac4-3761-478c-9f98-1bf95511b62e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.476610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc2ac4-3761-478c-9f98-1bf95511b62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:55.478739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc2ac4-3761-478c-9f98-1bf95511b62e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:55.589390 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1100/2184] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:55 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:55.593472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ba2b5-21f6-490c-86ff-c67fd8d216e4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:55.603651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ba2b5-21f6-490c-86ff-c67fd8d216e4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:55.608901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636450f-b1a9-45eb-930e-ecc8cb74a774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:55.609697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636450f-b1a9-45eb-930e-ecc8cb74a774 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.610053 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636450f-b1a9-45eb-930e-ecc8cb74a774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.610403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8636450f-b1a9-45eb-930e-ecc8cb74a774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.611212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636450f-b1a9-45eb-930e-ecc8cb74a774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:55.612914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8636450f-b1a9-45eb-930e-ecc8cb74a774 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:55.624874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1085/2185] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:55.630626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc2ac4-3761-478c-9f98-1bf95511b62e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:55.642915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc2ac4-3761-478c-9f98-1bf95511b62e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:55.648574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a83718-24ac-48e9-9e56-378f887374bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:55.649286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83718-24ac-48e9-9e56-378f887374bc admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.649578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83718-24ac-48e9-9e56-378f887374bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.649857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83718-24ac-48e9-9e56-378f887374bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.650584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a83718-24ac-48e9-9e56-378f887374bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:55.652174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a83718-24ac-48e9-9e56-378f887374bc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:55.663255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1101/2186] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:55.667117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636450f-b1a9-45eb-930e-ecc8cb74a774 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:55.678473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8636450f-b1a9-45eb-930e-ecc8cb74a774 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:55.683185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eecd4c-e233-4079-9eb4-4cf1a53d993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:55.684074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eecd4c-e233-4079-9eb4-4cf1a53d993e admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.684377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eecd4c-e233-4079-9eb4-4cf1a53d993e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.684643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eecd4c-e233-4079-9eb4-4cf1a53d993e admin admin] PATH_INFO: `/v3/projects/2e7ef2423ef84ebaa614aee81d51bbce/users/3a1121af30864cda8d8341728bcc1185/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.700499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eecd4c-e233-4079-9eb4-4cf1a53d993e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e7ef2423ef84ebaa614aee81d51bbce, user_id=3a1121af30864cda8d8341728bcc1185, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:55.703525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eecd4c-e233-4079-9eb4-4cf1a53d993e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:55.784321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1086/2187] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:55 2022] PUT /identity/v3/projects/2e7ef2423ef84ebaa614aee81d51bbce/users/3a1121af30864cda8d8341728bcc1185/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:55.795689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17491c-ca5f-46e0-b107-873f6f916350 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.796112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17491c-ca5f-46e0-b107-873f6f916350 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.796419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17491c-ca5f-46e0-b107-873f6f916350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.826715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ad17491c-ca5f-46e0-b107-873f6f916350 None None] MFA Rules not processed for user `3a1121af30864cda8d8341728bcc1185`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:55.827967 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ad17491c-ca5f-46e0-b107-873f6f916350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:55.901166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1102/2188] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:55 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:55.905941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eecd4c-e233-4079-9eb4-4cf1a53d993e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:55.951219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:55.977229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eecd4c-e233-4079-9eb4-4cf1a53d993e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:55.981961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bc736-e595-40c3-a3ff-40110d7a9085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:55.982608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bc736-e595-40c3-a3ff-40110d7a9085 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:55.982888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bc736-e595-40c3-a3ff-40110d7a9085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:55.983186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bc736-e595-40c3-a3ff-40110d7a9085 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:55.983880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bc736-e595-40c3-a3ff-40110d7a9085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:55.985867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bc736-e595-40c3-a3ff-40110d7a9085 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:56.071249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1087/2189] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:37:55 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:56.072630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17491c-ca5f-46e0-b107-873f6f916350 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:56.113356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17491c-ca5f-46e0-b107-873f6f916350 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:56.178757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007a801-1d17-4749-9193-caef2b5d37eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:56.179403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007a801-1d17-4749-9193-caef2b5d37eb admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:56.179692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007a801-1d17-4749-9193-caef2b5d37eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:56.179971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007a801-1d17-4749-9193-caef2b5d37eb admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:56.180659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007a801-1d17-4749-9193-caef2b5d37eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:56.182891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b007a801-1d17-4749-9193-caef2b5d37eb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:56.284707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1103/2190] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:37:56 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:37:56.290002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bc736-e595-40c3-a3ff-40110d7a9085 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:56.303585 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bc736-e595-40c3-a3ff-40110d7a9085 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:56.308727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175183c-09d8-4152-a93f-028842ee8780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:56.309510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175183c-09d8-4152-a93f-028842ee8780 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:56.309843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175183c-09d8-4152-a93f-028842ee8780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:56.310120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175183c-09d8-4152-a93f-028842ee8780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:56.311011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175183c-09d8-4152-a93f-028842ee8780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:56.312757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175183c-09d8-4152-a93f-028842ee8780 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:56.325133 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1088/2191] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:37:56.329245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007a801-1d17-4749-9193-caef2b5d37eb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:56.340176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007a801-1d17-4749-9193-caef2b5d37eb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:56.347494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97486d-9f6b-40b7-b8e2-bc48c9b86fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:56.347494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97486d-9f6b-40b7-b8e2-bc48c9b86fec admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:56.347494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97486d-9f6b-40b7-b8e2-bc48c9b86fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:56.347494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97486d-9f6b-40b7-b8e2-bc48c9b86fec admin admin] PATH_INFO: `/v3/projects/d1244610f09d4197b7e8f859c156ceab/users/f92d0c407887451c891022c41d48fb05/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:56.362537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97486d-9f6b-40b7-b8e2-bc48c9b86fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1244610f09d4197b7e8f859c156ceab, user_id=f92d0c407887451c891022c41d48fb05, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:56.370803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97486d-9f6b-40b7-b8e2-bc48c9b86fec admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:56.461266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1104/2192] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:56 2022] PUT /identity/v3/projects/d1244610f09d4197b7e8f859c156ceab/users/f92d0c407887451c891022c41d48fb05/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:56.465242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175183c-09d8-4152-a93f-028842ee8780 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:56.475418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175183c-09d8-4152-a93f-028842ee8780 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:56.538640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04b98-b609-4f06-b695-19a7a4770dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:56.539002 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04b98-b609-4f06-b695-19a7a4770dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:56.539298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04b98-b609-4f06-b695-19a7a4770dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:56.539593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04b98-b609-4f06-b695-19a7a4770dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:56.540466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04b98-b609-4f06-b695-19a7a4770dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:56.542400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04b98-b609-4f06-b695-19a7a4770dcc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:56.559564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1089/2193] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:37:56.561134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97486d-9f6b-40b7-b8e2-bc48c9b86fec admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:56.573076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97486d-9f6b-40b7-b8e2-bc48c9b86fec admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:56.579001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:56.579930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:56.580340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:56.580631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] PATH_INFO: `/v3/projects/d1244610f09d4197b7e8f859c156ceab/users/f92d0c407887451c891022c41d48fb05/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:56.582890 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1244610f09d4197b7e8f859c156ceab, user_id=f92d0c407887451c891022c41d48fb05, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:56.586353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:56.605960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1105/2194] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:37:56 2022] PUT /identity/v3/projects/d1244610f09d4197b7e8f859c156ceab/users/f92d0c407887451c891022c41d48fb05/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:37:56.621668 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe8cf2-20c7-4dda-9dd6-b165f940bca1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:56.621963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe8cf2-20c7-4dda-9dd6-b165f940bca1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:56.622269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe8cf2-20c7-4dda-9dd6-b165f940bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:56.644386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:56.661200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-25fe8cf2-20c7-4dda-9dd6-b165f940bca1 None None] MFA Rules not processed for user `f92d0c407887451c891022c41d48fb05`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:56.662695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-25fe8cf2-20c7-4dda-9dd6-b165f940bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:56.747471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1090/2195] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:37:56.762767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:56.764455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:56.827688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033375a-d395-4d58-ad76-89521d04863f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:56.831507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ea732-6e2c-4c52-a630-af08b66489b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:56.835763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ea732-6e2c-4c52-a630-af08b66489b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:56.835763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ea732-6e2c-4c52-a630-af08b66489b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:56.835763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ea732-6e2c-4c52-a630-af08b66489b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:56.842518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ea732-6e2c-4c52-a630-af08b66489b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:56.843969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ea732-6e2c-4c52-a630-af08b66489b8 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:56.853420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1106/2196] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:37:56.868343 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7a84a-d7fc-4a28-b3c0-d88448510de3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:56.868639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7a84a-d7fc-4a28-b3c0-d88448510de3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:56.868930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7a84a-d7fc-4a28-b3c0-d88448510de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:56.887015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f2c7a84a-d7fc-4a28-b3c0-d88448510de3 None None] MFA Rules not processed for user `9d420e4b554e4deab21b89094e0547ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:56.888220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f2c7a84a-d7fc-4a28-b3c0-d88448510de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:56.943661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1091/2197] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:57.076088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ea732-6e2c-4c52-a630-af08b66489b8 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:57.087172 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ea732-6e2c-4c52-a630-af08b66489b8 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:57.095026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f51db-b7b6-420a-947d-4e6b8dcb660a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:57.095981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f51db-b7b6-420a-947d-4e6b8dcb660a service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:57.096464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f51db-b7b6-420a-947d-4e6b8dcb660a service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:57.096956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f51db-b7b6-420a-947d-4e6b8dcb660a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:57.152735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:57.166941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f51db-b7b6-420a-947d-4e6b8dcb660a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:57.168858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f51db-b7b6-420a-947d-4e6b8dcb660a service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:57.217008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7a84a-d7fc-4a28-b3c0-d88448510de3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:57.218455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f2c7a84a-d7fc-4a28-b3c0-d88448510de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:57.227323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1107/2198] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:57.347057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7a84a-d7fc-4a28-b3c0-d88448510de3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:57.384308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:57.385672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:57.386001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:57.386289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] PATH_INFO: `/v3/users/33edd0878dc145ec874957b454d92e1f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:57.387583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33edd0878dc145ec874957b454d92e1f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:57.389719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:57.618710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33edd0878dc145ec874957b454d92e1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:57.649669 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33edd0878dc145ec874957b454d92e1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:57.650930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33edd0878dc145ec874957b454d92e1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:57.652338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33edd0878dc145ec874957b454d92e1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:57.774879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-c788a5f6-3d4d-4457-8d8d-6b8eb0f80691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33edd0878dc145ec874957b454d92e1f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:57.789587 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1092/2199] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:37:57 2022] DELETE /identity/v3/users/33edd0878dc145ec874957b454d92e1f => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:57.794630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bccfe-4864-40f2-98ff-87058931fe32 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:57.794997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bccfe-4864-40f2-98ff-87058931fe32 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:57.795327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bccfe-4864-40f2-98ff-87058931fe32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:57.833138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b83bccfe-4864-40f2-98ff-87058931fe32 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:57.835080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b83bccfe-4864-40f2-98ff-87058931fe32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:57.939357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1108/2200] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:57.996813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb18cb4-3e7c-464f-bebb-a92fdb9a9230 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:57.997155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb18cb4-3e7c-464f-bebb-a92fdb9a9230 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:57.997430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb18cb4-3e7c-464f-bebb-a92fdb9a9230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:58.002547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bccfe-4864-40f2-98ff-87058931fe32 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:58.013652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b83bccfe-4864-40f2-98ff-87058931fe32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:58.083132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1eb18cb4-3e7c-464f-bebb-a92fdb9a9230 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:58.083132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1eb18cb4-3e7c-464f-bebb-a92fdb9a9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:58.083814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:58.128005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bccfe-4864-40f2-98ff-87058931fe32 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:58.130829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:58.131173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:58.131338 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:58.131624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:58.154212 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:58.158140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:58.179138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1109/2201] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:37:58.183386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:37:58.252385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1093/2202] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:37:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:58.312840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:58.314314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:58.395506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7affa-e19b-4a20-a438-83578d76c330 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:58.402451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e2035-40f1-46d3-8b80-598b5930f2c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:58.403128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2035-40f1-46d3-8b80-598b5930f2c3 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:58.403408 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2035-40f1-46d3-8b80-598b5930f2c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:58.403682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e2035-40f1-46d3-8b80-598b5930f2c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:58.412818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e2035-40f1-46d3-8b80-598b5930f2c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:58.417798 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e2035-40f1-46d3-8b80-598b5930f2c3 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:58.422264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1110/2203] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:37:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:37:59.248424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb18cb4-3e7c-464f-bebb-a92fdb9a9230 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:59.249941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1eb18cb4-3e7c-464f-bebb-a92fdb9a9230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:59.280507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb18cb4-3e7c-464f-bebb-a92fdb9a9230 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:59.297667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:59.298904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:59.304847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:59.305426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] PATH_INFO: `/v3/projects/b3f7c1cdb98740d8a855fb84f66ce550` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:59.307855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f7c1cdb98740d8a855fb84f66ce550)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:59.310167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:37:59.456162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f7c1cdb98740d8a855fb84f66ce550', 'request_id': 'req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:59.464316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f7c1cdb98740d8a855fb84f66ce550', 'request_id': 'req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:59.475086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f7c1cdb98740d8a855fb84f66ce550', 'request_id': 'req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:59.642550 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] The token cache is being invalidate because project b3f7c1cdb98740d8a855fb84f66ce550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:37:59.644404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:59.646604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-92085a7c-4cb4-4ffa-ab31-3ad885b2eb40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:37:59.649253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1094/2204] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:37:59 2022] DELETE /identity/v3/projects/b3f7c1cdb98740d8a855fb84f66ce550 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:37:59.654130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d81b52-4741-48eb-8d94-369c21327de5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:59.657664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d81b52-4741-48eb-8d94-369c21327de5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:59.658504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d81b52-4741-48eb-8d94-369c21327de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:59.686961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-78d81b52-4741-48eb-8d94-369c21327de5 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:59.689207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-78d81b52-4741-48eb-8d94-369c21327de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:59.836392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1111/2205] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:59.848594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197f3d2-bcf9-4be0-bcae-e6cab96170c8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:59.854270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197f3d2-bcf9-4be0-bcae-e6cab96170c8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:59.854575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197f3d2-bcf9-4be0-bcae-e6cab96170c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:59.891153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b197f3d2-bcf9-4be0-bcae-e6cab96170c8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:37:59.893004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b197f3d2-bcf9-4be0-bcae-e6cab96170c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:37:59.915424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1095/2206] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:37:59.920475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d81b52-4741-48eb-8d94-369c21327de5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:37:59.943346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d81b52-4741-48eb-8d94-369c21327de5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:37:59.966726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:37:59.970876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:37:59.971171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:37:59.972094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:37:59.977707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:37:59.978322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:37:59.983283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:00.004150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1112/2207] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:38:00.009289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197f3d2-bcf9-4be0-bcae-e6cab96170c8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:00.025036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197f3d2-bcf9-4be0-bcae-e6cab96170c8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:00.056022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835ae39-028c-46c0-bd71-6ea3f8f3bc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:00.061232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835ae39-028c-46c0-bd71-6ea3f8f3bc74 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:00.061232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835ae39-028c-46c0-bd71-6ea3f8f3bc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:00.061232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835ae39-028c-46c0-bd71-6ea3f8f3bc74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:00.061232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835ae39-028c-46c0-bd71-6ea3f8f3bc74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:00.075264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835ae39-028c-46c0-bd71-6ea3f8f3bc74 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:00.388329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1096/2208] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:38:00 2022] POST /identity/v3/projects => generated 397 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:38:00.392615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:00.409578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779d856-8486-4bc8-9e4d-f3d79a9c3f3b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:00.553038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:00.602723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1bc12-4268-481c-8b38-ddacf99ace61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:00.603587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1bc12-4268-481c-8b38-ddacf99ace61 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:00.603946 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1bc12-4268-481c-8b38-ddacf99ace61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:00.604299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1bc12-4268-481c-8b38-ddacf99ace61 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:00.605061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1bc12-4268-481c-8b38-ddacf99ace61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:00.607493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1bc12-4268-481c-8b38-ddacf99ace61 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:00.724423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1113/2209] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:38:00 2022] POST /identity/v3/users => generated 299 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:38:00.728385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835ae39-028c-46c0-bd71-6ea3f8f3bc74 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:00.749307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835ae39-028c-46c0-bd71-6ea3f8f3bc74 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:00.762545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc29b49-1891-42d5-b63a-bd9e633a4ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:00.769252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc29b49-1891-42d5-b63a-bd9e633a4ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:00.769252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc29b49-1891-42d5-b63a-bd9e633a4ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:00.769252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc29b49-1891-42d5-b63a-bd9e633a4ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:00.769252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc29b49-1891-42d5-b63a-bd9e633a4ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:00.769252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc29b49-1891-42d5-b63a-bd9e633a4ed7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:00.789621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1097/2210] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:38:00.791285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1bc12-4268-481c-8b38-ddacf99ace61 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:00.803644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1bc12-4268-481c-8b38-ddacf99ace61 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:00.808245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bd099-748f-47dc-b455-3ee459c574a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:00.809253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd099-748f-47dc-b455-3ee459c574a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:00.809780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd099-748f-47dc-b455-3ee459c574a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:00.810490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd099-748f-47dc-b455-3ee459c574a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:00.822014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bd099-748f-47dc-b455-3ee459c574a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:00.826314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bd099-748f-47dc-b455-3ee459c574a2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:00.839115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1114/2211] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:38:00.843238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc29b49-1891-42d5-b63a-bd9e633a4ed7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:00.858378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc29b49-1891-42d5-b63a-bd9e633a4ed7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:00.863896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:00.864879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:00.865198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:00.865483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] PATH_INFO: `/v3/projects/92a4747ad25f422ca4727bd1e864d988/users/274588cb1b1d4a9594fd16b97649969a/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:00.885238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a4747ad25f422ca4727bd1e864d988, user_id=274588cb1b1d4a9594fd16b97649969a, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:00.888915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:01.022819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1098/2212] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:38:00 2022] PUT /identity/v3/projects/92a4747ad25f422ca4727bd1e864d988/users/274588cb1b1d4a9594fd16b97649969a/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:01.031091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36cc7-06e9-46f9-b325-f21e6226a8c1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:01.032276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36cc7-06e9-46f9-b325-f21e6226a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:01.032676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d36cc7-06e9-46f9-b325-f21e6226a8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:01.073981 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-36d36cc7-06e9-46f9-b325-f21e6226a8c1 None None] MFA Rules not processed for user `274588cb1b1d4a9594fd16b97649969a`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:01.076546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-36d36cc7-06e9-46f9-b325-f21e6226a8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:01.194839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1115/2213] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:01.196639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:01.198412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:01.304876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:01.321656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca1bcb-9586-46da-b640-483d723ca330 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:01.326974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe0e3f-a20e-4246-9b32-d85c1644611c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:01.327874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe0e3f-a20e-4246-9b32-d85c1644611c service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:01.328289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe0e3f-a20e-4246-9b32-d85c1644611c service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:01.328658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe0e3f-a20e-4246-9b32-d85c1644611c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:01.340833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe0e3f-a20e-4246-9b32-d85c1644611c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:01.342555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe0e3f-a20e-4246-9b32-d85c1644611c service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:01.379817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1099/2214] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:05.271377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d36cc7-06e9-46f9-b325-f21e6226a8c1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:05.272863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-36d36cc7-06e9-46f9-b325-f21e6226a8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:05.390160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d36cc7-06e9-46f9-b325-f21e6226a8c1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:05.395908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:05.398827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:05.399061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:05.399448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] PATH_INFO: `/v3/users/3a1121af30864cda8d8341728bcc1185` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:05.400883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a1121af30864cda8d8341728bcc1185)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:05.403010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:05.568115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a1121af30864cda8d8341728bcc1185'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:05.574595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:05.576851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a1121af30864cda8d8341728bcc1185'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:05.598933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a1121af30864cda8d8341728bcc1185'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:05.600046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a1121af30864cda8d8341728bcc1185'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:05.658012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a1121af30864cda8d8341728bcc1185'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:05.665793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1116/2215] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:38:05 2022] DELETE /identity/v3/users/3a1121af30864cda8d8341728bcc1185 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:05.670190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391492-fdd0-4952-a68a-cdee7bfa8152 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:05.670705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391492-fdd0-4952-a68a-cdee7bfa8152 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:05.670784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a391492-fdd0-4952-a68a-cdee7bfa8152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:05.697979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-5a391492-fdd0-4952-a68a-cdee7bfa8152 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:05.699731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5a391492-fdd0-4952-a68a-cdee7bfa8152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:05.792386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1100/2216] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:05.819399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:05.821130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:05.942105 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb06855-968f-46ef-a605-c4d94ac4308c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:05.949797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6cecf-70e3-4418-a337-1b4dceaa2c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:05.950711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6cecf-70e3-4418-a337-1b4dceaa2c9e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:05.951010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6cecf-70e3-4418-a337-1b4dceaa2c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:05.951297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6cecf-70e3-4418-a337-1b4dceaa2c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:05.976222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6cecf-70e3-4418-a337-1b4dceaa2c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:05.979641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6cecf-70e3-4418-a337-1b4dceaa2c9e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:05.993151 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1117/2217] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:06.590862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a391492-fdd0-4952-a68a-cdee7bfa8152 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:06.591806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5a391492-fdd0-4952-a68a-cdee7bfa8152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:06.609963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a391492-fdd0-4952-a68a-cdee7bfa8152 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:06.615317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:06.616290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:06.622080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:06.625620 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] PATH_INFO: `/v3/projects/2e7ef2423ef84ebaa614aee81d51bbce` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:06.631310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e7ef2423ef84ebaa614aee81d51bbce)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:06.633049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:06.747607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e7ef2423ef84ebaa614aee81d51bbce', 'request_id': 'req-b162f77d-479f-4aaa-8573-3ffc8414a4e8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:06.755556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e7ef2423ef84ebaa614aee81d51bbce', 'request_id': 'req-b162f77d-479f-4aaa-8573-3ffc8414a4e8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:06.761784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e7ef2423ef84ebaa614aee81d51bbce', 'request_id': 'req-b162f77d-479f-4aaa-8573-3ffc8414a4e8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:06.838141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] The token cache is being invalidate because project 2e7ef2423ef84ebaa614aee81d51bbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:38:06.838338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:06.839012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:06.840599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1101/2218] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:38:06 2022] DELETE /identity/v3/projects/2e7ef2423ef84ebaa614aee81d51bbce => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:06.844713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6cecf-70e3-4418-a337-1b4dceaa2c9e service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:06.855517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6cecf-70e3-4418-a337-1b4dceaa2c9e service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:06.920725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:06.921664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:06.922011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:06.922276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] PATH_INFO: `/v3/users/f92d0c407887451c891022c41d48fb05` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:06.923651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92d0c407887451c891022c41d48fb05)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:06.926209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:07.465266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92d0c407887451c891022c41d48fb05'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:07.482870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92d0c407887451c891022c41d48fb05'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:07.503197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92d0c407887451c891022c41d48fb05'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:07.504248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92d0c407887451c891022c41d48fb05'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:07.513664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:07.575332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-382571ad-a297-4e1a-a4b6-1b18e8f46eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92d0c407887451c891022c41d48fb05'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:07.578337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1118/2219] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:38:06 2022] DELETE /identity/v3/users/f92d0c407887451c891022c41d48fb05 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:08.109470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:08.111666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:08.131709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162f77d-479f-4aaa-8573-3ffc8414a4e8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:08.243384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:08.251540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:08.252005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:08.252424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] PATH_INFO: `/v3/projects/d1244610f09d4197b7e8f859c156ceab` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:08.254028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1244610f09d4197b7e8f859c156ceab)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:08.256121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:08.285027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:08.464061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1244610f09d4197b7e8f859c156ceab', 'request_id': 'req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:08.477923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1244610f09d4197b7e8f859c156ceab', 'request_id': 'req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:08.492869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1244610f09d4197b7e8f859c156ceab', 'request_id': 'req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:08.575642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] The token cache is being invalidate because project d1244610f09d4197b7e8f859c156ceab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:38:08.576087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:08.577008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f5a17631-cab2-4b42-91bf-a3aa85ec01d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:08.582596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1102/2220] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:38:08 2022] DELETE /identity/v3/projects/d1244610f09d4197b7e8f859c156ceab => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:08.584135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516a803-8f1d-471b-a453-afe2a8c69973 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:08.584658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516a803-8f1d-471b-a453-afe2a8c69973 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:08.584754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516a803-8f1d-471b-a453-afe2a8c69973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:08.611528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8516a803-8f1d-471b-a453-afe2a8c69973 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:08.612847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8516a803-8f1d-471b-a453-afe2a8c69973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:08.755292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1119/2221] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:38:08.756121 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e397e-eb1b-475a-b900-68dcc36f0039 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:08.756437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e397e-eb1b-475a-b900-68dcc36f0039 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:08.756716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e397e-eb1b-475a-b900-68dcc36f0039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:08.803782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-735e397e-eb1b-475a-b900-68dcc36f0039 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:08.810047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-735e397e-eb1b-475a-b900-68dcc36f0039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:08.829458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1103/2222] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:08.830277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516a803-8f1d-471b-a453-afe2a8c69973 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:08.842294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516a803-8f1d-471b-a453-afe2a8c69973 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:08.847840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:08.849479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:08.850005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:08.850501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:08.851439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:38:08.851962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:08.854060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:08.873473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1120/2223] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:38:08.877592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e397e-eb1b-475a-b900-68dcc36f0039 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:08.890715 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e397e-eb1b-475a-b900-68dcc36f0039 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:08.897212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147585a-103e-4356-932e-1292537cd73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:08.898046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147585a-103e-4356-932e-1292537cd73a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:08.898328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147585a-103e-4356-932e-1292537cd73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:08.898598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147585a-103e-4356-932e-1292537cd73a admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:08.899490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147585a-103e-4356-932e-1292537cd73a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:08.903038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147585a-103e-4356-932e-1292537cd73a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:09.083918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1104/2224] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:38:08 2022] POST /identity/v3/projects => generated 361 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:38:09.085918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:09.099824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990ad48-441c-4892-84f3-75ef41de4e74 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:09.213237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f96d2b-1af6-45b0-92ad-2c785590160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:09.214238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f96d2b-1af6-45b0-92ad-2c785590160e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:09.214705 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f96d2b-1af6-45b0-92ad-2c785590160e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:09.215137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f96d2b-1af6-45b0-92ad-2c785590160e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:09.216035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f96d2b-1af6-45b0-92ad-2c785590160e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:09.219031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f96d2b-1af6-45b0-92ad-2c785590160e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:09.296398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1121/2225] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:38:09 2022] POST /identity/v3/users => generated 281 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:38:09.300508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147585a-103e-4356-932e-1292537cd73a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:09.315467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147585a-103e-4356-932e-1292537cd73a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:09.328054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5f76d-6abc-4332-af79-9527c59aa2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:09.328744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5f76d-6abc-4332-af79-9527c59aa2cc admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:09.329024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5f76d-6abc-4332-af79-9527c59aa2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:09.330117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5f76d-6abc-4332-af79-9527c59aa2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:09.330117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5f76d-6abc-4332-af79-9527c59aa2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:09.331655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5f76d-6abc-4332-af79-9527c59aa2cc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:09.369520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1105/2226] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:38:09.373629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f96d2b-1af6-45b0-92ad-2c785590160e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:09.386099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f96d2b-1af6-45b0-92ad-2c785590160e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:09.396844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922964e8-64e4-4b1a-968e-008c96d89197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:09.398118 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922964e8-64e4-4b1a-968e-008c96d89197 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:09.398428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922964e8-64e4-4b1a-968e-008c96d89197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:09.398712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922964e8-64e4-4b1a-968e-008c96d89197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:09.399459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922964e8-64e4-4b1a-968e-008c96d89197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:09.401430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922964e8-64e4-4b1a-968e-008c96d89197 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:09.416150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1122/2227] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:38:09.426682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5f76d-6abc-4332-af79-9527c59aa2cc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:09.449056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5f76d-6abc-4332-af79-9527c59aa2cc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:09.456270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:09.457810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:09.458005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:09.459178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] PATH_INFO: `/v3/projects/adf05d01fd4d4403b305104f165a1688/users/da29c576a55249f694132012fde0279d/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:09.483208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf05d01fd4d4403b305104f165a1688, user_id=da29c576a55249f694132012fde0279d, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:09.486512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:09.600841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1106/2228] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:38:09 2022] PUT /identity/v3/projects/adf05d01fd4d4403b305104f165a1688/users/da29c576a55249f694132012fde0279d/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:09.603070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a2d5c-46cf-41df-8256-236c76122d65 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:09.603533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a2d5c-46cf-41df-8256-236c76122d65 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:09.603700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a2d5c-46cf-41df-8256-236c76122d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:09.651060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f74a2d5c-46cf-41df-8256-236c76122d65 None None] MFA Rules not processed for user `da29c576a55249f694132012fde0279d`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:09.652555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f74a2d5c-46cf-41df-8256-236c76122d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:09.806435 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:09.848665 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1123/2229] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:09.901714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:09.906602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:10.022097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad74a-a820-4967-942f-fdbab2af4fc1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:10.028339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbabc9-1378-447c-93e9-d44b2fdfea33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:10.029038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbabc9-1378-447c-93e9-d44b2fdfea33 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:10.030078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbabc9-1378-447c-93e9-d44b2fdfea33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:10.030664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbabc9-1378-447c-93e9-d44b2fdfea33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:10.057570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbabc9-1378-447c-93e9-d44b2fdfea33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:10.059120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbabc9-1378-447c-93e9-d44b2fdfea33 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:10.080781 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1107/2230] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:10.099626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a2d5c-46cf-41df-8256-236c76122d65 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:10.280718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a2d5c-46cf-41df-8256-236c76122d65 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:10.287472 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:10.292021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:10.292021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:10.292021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] PATH_INFO: `/v3/users/da29c576a55249f694132012fde0279d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:10.292021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da29c576a55249f694132012fde0279d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:10.292411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:10.611699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da29c576a55249f694132012fde0279d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:10.623938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da29c576a55249f694132012fde0279d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:10.643111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da29c576a55249f694132012fde0279d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:10.644011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da29c576a55249f694132012fde0279d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:10.717903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da29c576a55249f694132012fde0279d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:10.726160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1124/2231] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:38:10 2022] DELETE /identity/v3/users/da29c576a55249f694132012fde0279d => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:10.730958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bfbc0-407e-430e-ada0-8006ec5a4c24 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:10.731997 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bfbc0-407e-430e-ada0-8006ec5a4c24 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:10.732441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bfbc0-407e-430e-ada0-8006ec5a4c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:10.759449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6d6bfbc0-407e-430e-ada0-8006ec5a4c24 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:10.760885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6d6bfbc0-407e-430e-ada0-8006ec5a4c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:10.816365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:10.865788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1108/2232] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:10.880901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:10.884517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:10.997801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ab29-fc65-422c-a17e-f7ffc8debf71 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:11.003709 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ab7ef-f283-4335-9ce5-a4a8f7d71cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:11.005352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab7ef-f283-4335-9ce5-a4a8f7d71cae service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:11.006601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab7ef-f283-4335-9ce5-a4a8f7d71cae service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:11.007020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ab7ef-f283-4335-9ce5-a4a8f7d71cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:11.030550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ab7ef-f283-4335-9ce5-a4a8f7d71cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:11.031643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ab7ef-f283-4335-9ce5-a4a8f7d71cae service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:11.050183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1125/2233] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:11.156638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bfbc0-407e-430e-ada0-8006ec5a4c24 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:11.158451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6d6bfbc0-407e-430e-ada0-8006ec5a4c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:11.181534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bfbc0-407e-430e-ada0-8006ec5a4c24 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:11.194163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:11.195485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:11.195888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:11.196437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] PATH_INFO: `/v3/users/274588cb1b1d4a9594fd16b97649969a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:11.197618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] RBAC: Authorizing `identity:delete_user(user_id=274588cb1b1d4a9594fd16b97649969a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:11.200170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:11.399268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '274588cb1b1d4a9594fd16b97649969a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:11.408868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a76b8-5f05-4d30-bb9b-115de319174f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:11.409213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a76b8-5f05-4d30-bb9b-115de319174f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:11.409509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a76b8-5f05-4d30-bb9b-115de319174f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:11.434012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '274588cb1b1d4a9594fd16b97649969a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:11.438575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-da9a76b8-5f05-4d30-bb9b-115de319174f None None] MFA Rules not processed for user `b6709eed72ca46468fd298929ae0f764`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:11.438575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-da9a76b8-5f05-4d30-bb9b-115de319174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:11.442945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:11.447652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '274588cb1b1d4a9594fd16b97649969a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:11.449683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '274588cb1b1d4a9594fd16b97649969a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:11.515940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1126/2234] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:11.524119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '274588cb1b1d4a9594fd16b97649969a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:11.542657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1109/2235] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:38:11 2022] DELETE /identity/v3/users/274588cb1b1d4a9594fd16b97649969a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:38:11.547120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6ca3-c4cc-415c-bb34-1a5197f4d59e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:11.551231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6ca3-c4cc-415c-bb34-1a5197f4d59e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:11.551231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f6ca3-c4cc-415c-bb34-1a5197f4d59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:11.618337 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e95f6ca3-c4cc-415c-bb34-1a5197f4d59e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:11.620971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e95f6ca3-c4cc-415c-bb34-1a5197f4d59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:11.730718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1127/2236] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:11.740906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:11.742475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:11.908044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf972e4f-2469-4be4-ae07-9e5874eb982c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:11.927932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50067bac-7f9d-4087-a227-0940e12f2e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:11.928667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50067bac-7f9d-4087-a227-0940e12f2e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:11.928902 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50067bac-7f9d-4087-a227-0940e12f2e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:11.929204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50067bac-7f9d-4087-a227-0940e12f2e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:11.963385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50067bac-7f9d-4087-a227-0940e12f2e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:11.965688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50067bac-7f9d-4087-a227-0940e12f2e4f service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:11.991739 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1110/2237] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:12.036469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f6ca3-c4cc-415c-bb34-1a5197f4d59e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:12.042632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e95f6ca3-c4cc-415c-bb34-1a5197f4d59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:12.062609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f6ca3-c4cc-415c-bb34-1a5197f4d59e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:12.068149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:12.069017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:12.069658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:12.069658 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] PATH_INFO: `/v3/projects/adf05d01fd4d4403b305104f165a1688` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:12.070937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf05d01fd4d4403b305104f165a1688)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:12.073368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:12.586541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf05d01fd4d4403b305104f165a1688', 'request_id': 'req-d9f24089-b31f-499d-9313-42fd28375a92'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:12.595500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf05d01fd4d4403b305104f165a1688', 'request_id': 'req-d9f24089-b31f-499d-9313-42fd28375a92'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:12.618791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf05d01fd4d4403b305104f165a1688', 'request_id': 'req-d9f24089-b31f-499d-9313-42fd28375a92'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:12.720826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] The token cache is being invalidate because project adf05d01fd4d4403b305104f165a1688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:38:12.721140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:12.722350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d9f24089-b31f-499d-9313-42fd28375a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:12.728234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1128/2238] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:38:12 2022] DELETE /identity/v3/projects/adf05d01fd4d4403b305104f165a1688 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:12.733349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0b6e4-0043-4b71-b8a2-155ee801195c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:12.733648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0b6e4-0043-4b71-b8a2-155ee801195c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:12.733935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0b6e4-0043-4b71-b8a2-155ee801195c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:12.774556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f3d0b6e4-0043-4b71-b8a2-155ee801195c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:12.776058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f3d0b6e4-0043-4b71-b8a2-155ee801195c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:12.885131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:12.908864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1111/2239] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:12.917861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453579-26b9-45e5-b640-83fabb523a25 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:12.918131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453579-26b9-45e5-b640-83fabb523a25 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:12.918581 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453579-26b9-45e5-b640-83fabb523a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:12.968325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0c453579-26b9-45e5-b640-83fabb523a25 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:12.969971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0c453579-26b9-45e5-b640-83fabb523a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:13.013255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1129/2240] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:13.013925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0b6e4-0043-4b71-b8a2-155ee801195c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:13.025143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0b6e4-0043-4b71-b8a2-155ee801195c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:13.031117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:13.032012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:13.032448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:13.032874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:13.033795 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:38:13.034273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:13.036164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:13.062574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1112/2241] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:38:13.067518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453579-26b9-45e5-b640-83fabb523a25 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:13.122031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:13.124810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453579-26b9-45e5-b640-83fabb523a25 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:13.124952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:13.139904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fcd2f-45ae-4c83-b46e-6a49ea1aa53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:13.140637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fcd2f-45ae-4c83-b46e-6a49ea1aa53f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:13.140917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fcd2f-45ae-4c83-b46e-6a49ea1aa53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:13.143888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967fcd2f-45ae-4c83-b46e-6a49ea1aa53f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:13.146924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fcd2f-45ae-4c83-b46e-6a49ea1aa53f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:13.158422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967fcd2f-45ae-4c83-b46e-6a49ea1aa53f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:13.176657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf3c9-33f3-4d0c-946b-b1d9777d77ee admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:13.183384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:13.184554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:13.185196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:13.186148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] PATH_INFO: `/v3/projects/92a4747ad25f422ca4727bd1e864d988` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:13.190941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a4747ad25f422ca4727bd1e864d988)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:13.190941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:13.284829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1130/2242] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:38:13 2022] POST /identity/v3/projects => generated 383 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:38:13.286063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fcd2f-45ae-4c83-b46e-6a49ea1aa53f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:13.313891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:13.339872 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a4747ad25f422ca4727bd1e864d988', 'request_id': 'req-67a335c5-013a-4f9c-af93-0a9d912387a1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:13.348892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a4747ad25f422ca4727bd1e864d988', 'request_id': 'req-67a335c5-013a-4f9c-af93-0a9d912387a1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:13.358437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a4747ad25f422ca4727bd1e864d988', 'request_id': 'req-67a335c5-013a-4f9c-af93-0a9d912387a1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:13.409178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967fcd2f-45ae-4c83-b46e-6a49ea1aa53f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:13.470927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] The token cache is being invalidate because project 92a4747ad25f422ca4727bd1e864d988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:38:13.471231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:13.472560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:38:13.474509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1113/2243] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:38:13 2022] DELETE /identity/v3/projects/92a4747ad25f422ca4727bd1e864d988 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:13.510003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8a9a3-9281-494c-aa0d-1570967480c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:13.511138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8a9a3-9281-494c-aa0d-1570967480c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:13.511586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8a9a3-9281-494c-aa0d-1570967480c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:13.512772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8a9a3-9281-494c-aa0d-1570967480c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:13.514176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8a9a3-9281-494c-aa0d-1570967480c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:13.518982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8a9a3-9281-494c-aa0d-1570967480c2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:13.656958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1131/2244] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:38:13 2022] POST /identity/v3/users => generated 292 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:38:13.671578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:13.674546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:13.699434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a335c5-013a-4f9c-af93-0a9d912387a1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:13.709686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d600af-ebef-424b-9e2d-8e9291dc199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:13.710433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d600af-ebef-424b-9e2d-8e9291dc199c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:13.710766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d600af-ebef-424b-9e2d-8e9291dc199c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:13.710996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d600af-ebef-424b-9e2d-8e9291dc199c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:13.711737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d600af-ebef-424b-9e2d-8e9291dc199c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:13.713404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d600af-ebef-424b-9e2d-8e9291dc199c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:13.730362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1114/2245] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:38:13.734616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8a9a3-9281-494c-aa0d-1570967480c2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:13.757221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8a9a3-9281-494c-aa0d-1570967480c2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:13.764275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5529e-99fd-49e0-8920-37e5a51555be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:13.767098 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5529e-99fd-49e0-8920-37e5a51555be admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:13.767388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5529e-99fd-49e0-8920-37e5a51555be admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:13.771125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5529e-99fd-49e0-8920-37e5a51555be admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:13.771125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5529e-99fd-49e0-8920-37e5a51555be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:13.772193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5529e-99fd-49e0-8920-37e5a51555be admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:13.787380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1132/2246] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:38:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:38:13.792268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d600af-ebef-424b-9e2d-8e9291dc199c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:13.806411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d600af-ebef-424b-9e2d-8e9291dc199c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:13.816083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:13.817818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:13.819280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:13.819584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] PATH_INFO: `/v3/projects/e302a2cd63ba46d7a6cef952ddb89242/users/af5fa544bc6440c9830d25e4ad4d5987/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:13.838606 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e302a2cd63ba46d7a6cef952ddb89242, user_id=af5fa544bc6440c9830d25e4ad4d5987, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:13.841379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:13.908624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1115/2247] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:38:13 2022] PUT /identity/v3/projects/e302a2cd63ba46d7a6cef952ddb89242/users/af5fa544bc6440c9830d25e4ad4d5987/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:38:13.914072 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0905a0-b2f5-4df0-9098-b543f1cfc2d3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:13.914651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0905a0-b2f5-4df0-9098-b543f1cfc2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:13.915224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0905a0-b2f5-4df0-9098-b543f1cfc2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:13.941423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ac0905a0-b2f5-4df0-9098-b543f1cfc2d3 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:13.941423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ac0905a0-b2f5-4df0-9098-b543f1cfc2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:14.008647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1133/2248] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:14.020310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:14.022362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:14.099655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f268-85e9-4c5a-a0dd-54a787c0e716 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:14.103447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:14.104099 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:14.104387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:14.104667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:14.114534 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:14.115987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:14.126506 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1116/2249] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:15.406271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82d0b-c00c-45f9-810b-c605d9c7146e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:15.406804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82d0b-c00c-45f9-810b-c605d9c7146e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:15.406894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a82d0b-c00c-45f9-810b-c605d9c7146e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:15.461225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-13a82d0b-c00c-45f9-810b-c605d9c7146e None None] MFA Rules not processed for user `af5fa544bc6440c9830d25e4ad4d5987`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:15.463065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-13a82d0b-c00c-45f9-810b-c605d9c7146e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:15.501906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:15.557180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1134/2250] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:38:15.572498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:15.574307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:15.616489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:15.642249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f884d5-a75a-4e03-a5fd-3798d5a76a41 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:15.647599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b9fff-28f4-4860-8a6b-84d743d38212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:15.648555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b9fff-28f4-4860-8a6b-84d743d38212 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:15.648940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b9fff-28f4-4860-8a6b-84d743d38212 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:15.649315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b9fff-28f4-4860-8a6b-84d743d38212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:15.659070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b9fff-28f4-4860-8a6b-84d743d38212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:15.661086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b9fff-28f4-4860-8a6b-84d743d38212 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:15.672355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1117/2251] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:17.352232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12751-fb9b-4048-ae7a-cf854ee8ee42 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:17.352787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12751-fb9b-4048-ae7a-cf854ee8ee42 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:17.352787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12751-fb9b-4048-ae7a-cf854ee8ee42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:17.375558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1da12751-fb9b-4048-ae7a-cf854ee8ee42 None None] MFA Rules not processed for user `af5fa544bc6440c9830d25e4ad4d5987`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:17.377273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1da12751-fb9b-4048-ae7a-cf854ee8ee42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:17.388008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1135/2252] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:23.453986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd0bd6-3415-4b58-a289-c604fa7585dd None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:23.454493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd0bd6-3415-4b58-a289-c604fa7585dd None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:23.454618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd0bd6-3415-4b58-a289-c604fa7585dd None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:23.456702 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1118/2253] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:38:23.468603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5bb4-0b37-474f-8bf0-2be6fad5f7aa None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:23.469082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5bb4-0b37-474f-8bf0-2be6fad5f7aa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:23.469211 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5bb4-0b37-474f-8bf0-2be6fad5f7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:23.551018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b1ca5bb4-0b37-474f-8bf0-2be6fad5f7aa None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:23.554226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b1ca5bb4-0b37-474f-8bf0-2be6fad5f7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:23.681500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1136/2254] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:38:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:23.697931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd0bd6-3415-4b58-a289-c604fa7585dd None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:23.702602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-39cd0bd6-3415-4b58-a289-c604fa7585dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:23.795858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd0bd6-3415-4b58-a289-c604fa7585dd None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:23.804346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeebc0b-8652-4d4b-bd81-1cf8c8274869 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47589bf9e7c4acba200979e3a91a6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:23.805079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeebc0b-8652-4d4b-bd81-1cf8c8274869 service glance] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:23.805418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeebc0b-8652-4d4b-bd81-1cf8c8274869 service glance] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:23.805697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeebc0b-8652-4d4b-bd81-1cf8c8274869 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:23.822259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeebc0b-8652-4d4b-bd81-1cf8c8274869 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:23.823811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeebc0b-8652-4d4b-bd81-1cf8c8274869 service glance] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:23.840109 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1119/2255] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:24.230204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f451b87-274d-440c-886e-80d4061d1eb6 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:24.230661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f451b87-274d-440c-886e-80d4061d1eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:24.230925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f451b87-274d-440c-886e-80d4061d1eb6 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:24.232359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1137/2256] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:38:24.237134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d15664-8c00-492d-a5b9-2f938143ad23 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:24.237430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d15664-8c00-492d-a5b9-2f938143ad23 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:24.237713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d15664-8c00-492d-a5b9-2f938143ad23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:24.256899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a2d15664-8c00-492d-a5b9-2f938143ad23 None None] MFA Rules not processed for user `1dd9a0e75308460080902d1da74074c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:24.258323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a2d15664-8c00-492d-a5b9-2f938143ad23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:24.265554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1120/2257] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:38:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:24.280575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f451b87-274d-440c-886e-80d4061d1eb6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:24.295043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f451b87-274d-440c-886e-80d4061d1eb6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:24.298803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735d710-48bd-4aaf-abaa-c274502941ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:24.299723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735d710-48bd-4aaf-abaa-c274502941ec service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:24.300019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735d710-48bd-4aaf-abaa-c274502941ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:24.300302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735d710-48bd-4aaf-abaa-c274502941ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:24.308283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735d710-48bd-4aaf-abaa-c274502941ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:24.309756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735d710-48bd-4aaf-abaa-c274502941ec service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:24.319376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1138/2258] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:24.405479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf9430-dac4-415e-b20f-a599083f9a48 None None] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:24.405853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf9430-dac4-415e-b20f-a599083f9a48 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:24.406209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf9430-dac4-415e-b20f-a599083f9a48 None None] PATH_INFO: `/` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:24.407776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1121/2259] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:38:24.413972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc1e89-8fb8-4f81-8649-fed294e5f908 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:24.414336 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc1e89-8fb8-4f81-8649-fed294e5f908 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:24.414685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc1e89-8fb8-4f81-8649-fed294e5f908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:24.432712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d3dc1e89-8fb8-4f81-8649-fed294e5f908 None None] MFA Rules not processed for user `f7ebbd8eb36c4919aedd3498bd8bc483`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:24.434039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d3dc1e89-8fb8-4f81-8649-fed294e5f908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:24.443823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1139/2260] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:38:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:24.454222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf9430-dac4-415e-b20f-a599083f9a48 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:24.465563 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf9430-dac4-415e-b20f-a599083f9a48 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:24.469320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211c8e9-4233-4d58-87af-d110fb319622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:24.469950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c8e9-4233-4d58-87af-d110fb319622 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:24.470261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c8e9-4233-4d58-87af-d110fb319622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:24.470536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c8e9-4233-4d58-87af-d110fb319622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:24.481471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211c8e9-4233-4d58-87af-d110fb319622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:24.482938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211c8e9-4233-4d58-87af-d110fb319622 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:24.494574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1122/2261] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:24.719579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd558aa-f5fa-4f92-88fa-b71ae86ea47f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:24.720605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd558aa-f5fa-4f92-88fa-b71ae86ea47f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:24.721070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd558aa-f5fa-4f92-88fa-b71ae86ea47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:24.757056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1fd558aa-f5fa-4f92-88fa-b71ae86ea47f None None] MFA Rules not processed for user `db802295be064253ae59efcbfd12be8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:24.758335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1fd558aa-f5fa-4f92-88fa-b71ae86ea47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:24.881765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1140/2262] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:38:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:30.019949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc35c8e-c731-4568-b937-a6ac62ee6796 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:30.020423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc35c8e-c731-4568-b937-a6ac62ee6796 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:30.020518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc35c8e-c731-4568-b937-a6ac62ee6796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:30.084208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7dc35c8e-c731-4568-b937-a6ac62ee6796 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:30.085407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7dc35c8e-c731-4568-b937-a6ac62ee6796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:30.177454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1123/2263] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:38:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:30.191436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd558aa-f5fa-4f92-88fa-b71ae86ea47f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:30.191436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1fd558aa-f5fa-4f92-88fa-b71ae86ea47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:30.194863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:30.294887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd558aa-f5fa-4f92-88fa-b71ae86ea47f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:30.300617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ab402-f1ee-4c31-9847-8f424ad4d4af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:30.301928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab402-f1ee-4c31-9847-8f424ad4d4af service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:30.302225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab402-f1ee-4c31-9847-8f424ad4d4af service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:30.302508 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ab402-f1ee-4c31-9847-8f424ad4d4af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:30.310836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab402-f1ee-4c31-9847-8f424ad4d4af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:30.312504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ab402-f1ee-4c31-9847-8f424ad4d4af service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:30.320564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1141/2264] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:38:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:38:40.173275 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:40.317621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=39 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:38:51.637030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc35c8e-c731-4568-b937-a6ac62ee6796 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:51.640992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7dc35c8e-c731-4568-b937-a6ac62ee6796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:51.743616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc35c8e-c731-4568-b937-a6ac62ee6796 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:51.749016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:51.751424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:51.751424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:51.751424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:51.752312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:51.764332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:51.765960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:51.775924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1124/2265] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:38:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:38:57.806936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160ef7-225a-4b73-a9a0-bcceb3fab51b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:57.807380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160ef7-225a-4b73-a9a0-bcceb3fab51b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:57.807599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d160ef7-225a-4b73-a9a0-bcceb3fab51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:57.829231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4d160ef7-225a-4b73-a9a0-bcceb3fab51b None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:38:57.830836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4d160ef7-225a-4b73-a9a0-bcceb3fab51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:38:57.838611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1142/2266] 10.222.0.41 () {58 vars in 967 bytes} [Sat May 28 03:38:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:38:57.862136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:38:57.874347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98faa45a-c03c-4eee-9d3a-db7e1fbe6c28 service placement] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:38:57.879573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd007f43-64a9-4938-9cf4-169e03ef216c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:38:57.880920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd007f43-64a9-4938-9cf4-169e03ef216c service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:38:57.881307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd007f43-64a9-4938-9cf4-169e03ef216c service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:38:57.881594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd007f43-64a9-4938-9cf4-169e03ef216c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:38:57.890090 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd007f43-64a9-4938-9cf4-169e03ef216c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:38:57.891688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd007f43-64a9-4938-9cf4-169e03ef216c service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:38:57.900226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1125/2267] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:39:04.489405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e453997-64b8-4b90-a5da-6df7cbac1c7e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:04.490170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e453997-64b8-4b90-a5da-6df7cbac1c7e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:04.490261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e453997-64b8-4b90-a5da-6df7cbac1c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:04.508309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6e453997-64b8-4b90-a5da-6df7cbac1c7e None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:39:04.509525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6e453997-64b8-4b90-a5da-6df7cbac1c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:04.516535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1143/2268] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:39:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:39:04.528720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd007f43-64a9-4938-9cf4-169e03ef216c service swift] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:04.539074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd007f43-64a9-4938-9cf4-169e03ef216c service swift] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:04.543995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef8aa9-4cc0-4787-b90c-5669fd6ad306 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:04.544619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef8aa9-4cc0-4787-b90c-5669fd6ad306 service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:04.544903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef8aa9-4cc0-4787-b90c-5669fd6ad306 service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:04.545219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef8aa9-4cc0-4787-b90c-5669fd6ad306 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:04.553164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef8aa9-4cc0-4787-b90c-5669fd6ad306 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:04.554738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef8aa9-4cc0-4787-b90c-5669fd6ad306 service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:04.562931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1126/2269] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:39:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 03:39:14.503079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:14.560126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=53 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:23.533280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82c3e2-35c6-436a-81b2-11546ba446b6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:23.533740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82c3e2-35c6-436a-81b2-11546ba446b6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:23.533873 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82c3e2-35c6-436a-81b2-11546ba446b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:23.551100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2f82c3e2-35c6-436a-81b2-11546ba446b6 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:39:23.552270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2f82c3e2-35c6-436a-81b2-11546ba446b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:23.559800 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1144/2270] 10.222.0.41 () {58 vars in 967 bytes} [Sat May 28 03:39:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:39:23.572634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef8aa9-4cc0-4787-b90c-5669fd6ad306 service swift] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:23.584262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef8aa9-4cc0-4787-b90c-5669fd6ad306 service swift] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:23.589391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa788-0cea-43ee-a2fc-fecf0b9b49a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:23.590026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa788-0cea-43ee-a2fc-fecf0b9b49a7 service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:23.590335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa788-0cea-43ee-a2fc-fecf0b9b49a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:23.590613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa788-0cea-43ee-a2fc-fecf0b9b49a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:23.598580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa788-0cea-43ee-a2fc-fecf0b9b49a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:23.600152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fa788-0cea-43ee-a2fc-fecf0b9b49a7 service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:23.608260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1127/2271] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:39:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:39:28.749345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82c3e2-35c6-436a-81b2-11546ba446b6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:28.751122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2f82c3e2-35c6-436a-81b2-11546ba446b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:28.768009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82c3e2-35c6-436a-81b2-11546ba446b6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:28.772607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:28.773526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:28.773813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:28.774089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] PATH_INFO: `/v3/users/db802295be064253ae59efcbfd12be8f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:28.775332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=db802295be064253ae59efcbfd12be8f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:28.782750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:28.925040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db802295be064253ae59efcbfd12be8f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:28.932861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db802295be064253ae59efcbfd12be8f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:28.944850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db802295be064253ae59efcbfd12be8f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:28.945609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db802295be064253ae59efcbfd12be8f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:28.985201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-db70b6da-3fca-4366-908f-bc27e1787eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db802295be064253ae59efcbfd12be8f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:28.991434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1145/2272] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:39:28 2022] DELETE /identity/v3/users/db802295be064253ae59efcbfd12be8f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:39:29.215779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa788-0cea-43ee-a2fc-fecf0b9b49a7 service swift] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:29.226408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fa788-0cea-43ee-a2fc-fecf0b9b49a7 service swift] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:29.283762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:29.284679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:29.284964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:29.285277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] PATH_INFO: `/v3/projects/ccf50ff938ce4fdb8d24871054e0015d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:29.286437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf50ff938ce4fdb8d24871054e0015d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:29.288321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:29.402709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf50ff938ce4fdb8d24871054e0015d', 'request_id': 'req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:29.408637 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf50ff938ce4fdb8d24871054e0015d', 'request_id': 'req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:29.414254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf50ff938ce4fdb8d24871054e0015d', 'request_id': 'req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:29.517003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] The token cache is being invalidate because project ccf50ff938ce4fdb8d24871054e0015d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:39:29.517321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:29.517931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-72f0e16b-e6da-4d39-8fa3-d6014d6e8f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:39:29.519450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1128/2273] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:39:29 2022] DELETE /identity/v3/projects/ccf50ff938ce4fdb8d24871054e0015d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:39:29.524769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5747c45-5b48-4db1-8a27-64006dadaa1b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:29.525061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5747c45-5b48-4db1-8a27-64006dadaa1b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:29.525366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5747c45-5b48-4db1-8a27-64006dadaa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:29.543953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e5747c45-5b48-4db1-8a27-64006dadaa1b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:39:29.545231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e5747c45-5b48-4db1-8a27-64006dadaa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:29.602683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1146/2274] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:39:29.610599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c6cdc-61e7-4fde-8427-e2e111555294 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:29.610925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c6cdc-61e7-4fde-8427-e2e111555294 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:29.611208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c6cdc-61e7-4fde-8427-e2e111555294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:29.630806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-cc4c6cdc-61e7-4fde-8427-e2e111555294 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:39:29.632020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cc4c6cdc-61e7-4fde-8427-e2e111555294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:29.639937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1129/2275] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:39:29.643979 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5747c45-5b48-4db1-8a27-64006dadaa1b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:29.654081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5747c45-5b48-4db1-8a27-64006dadaa1b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:29.658598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:29.659242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:29.659530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:29.659811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:29.660540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:39:29.660851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:29.662495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:29.674474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1147/2276] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:39:29.683679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c6cdc-61e7-4fde-8427-e2e111555294 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:29.693214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c6cdc-61e7-4fde-8427-e2e111555294 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:29.698482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8703be0-93ec-4c28-ae8a-757e0d8c5979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:29.699130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8703be0-93ec-4c28-ae8a-757e0d8c5979 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:29.699413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8703be0-93ec-4c28-ae8a-757e0d8c5979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:29.699690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8703be0-93ec-4c28-ae8a-757e0d8c5979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:29.700378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8703be0-93ec-4c28-ae8a-757e0d8c5979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:29.702357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8703be0-93ec-4c28-ae8a-757e0d8c5979 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:29.787069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1130/2277] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:39:29 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:39:29.791253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:29.801182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f03755-d482-4856-95f2-a2b12c9735bb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:29.847876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:29.856420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322e401-5d05-4378-a870-ceec5fd66256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:29.857076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322e401-5d05-4378-a870-ceec5fd66256 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:29.862195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322e401-5d05-4378-a870-ceec5fd66256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:29.862195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322e401-5d05-4378-a870-ceec5fd66256 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:29.862195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322e401-5d05-4378-a870-ceec5fd66256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:29.862195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322e401-5d05-4378-a870-ceec5fd66256 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:29.966961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1148/2278] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:39:29 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:39:29.968895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8703be0-93ec-4c28-ae8a-757e0d8c5979 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:29.979905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8703be0-93ec-4c28-ae8a-757e0d8c5979 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:29.984627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df71a9b-cc39-4772-af17-a70a22e81b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:29.985356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df71a9b-cc39-4772-af17-a70a22e81b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:29.985651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df71a9b-cc39-4772-af17-a70a22e81b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:29.985925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df71a9b-cc39-4772-af17-a70a22e81b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:29.986673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df71a9b-cc39-4772-af17-a70a22e81b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:29.988256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df71a9b-cc39-4772-af17-a70a22e81b8e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:29.999439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1131/2279] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:39:30.003386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322e401-5d05-4378-a870-ceec5fd66256 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:30.014978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322e401-5d05-4378-a870-ceec5fd66256 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:30.019271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408b596-db95-4763-afa6-92208034379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:30.019930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408b596-db95-4763-afa6-92208034379b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:30.020217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408b596-db95-4763-afa6-92208034379b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:30.020500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408b596-db95-4763-afa6-92208034379b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:30.021259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408b596-db95-4763-afa6-92208034379b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:30.022861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408b596-db95-4763-afa6-92208034379b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:30.033909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1149/2280] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:39:30.038082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df71a9b-cc39-4772-af17-a70a22e81b8e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:30.047857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df71a9b-cc39-4772-af17-a70a22e81b8e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:30.052488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:30.053429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:30.053712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:30.053991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] PATH_INFO: `/v3/projects/8aaa4d7bd49a4347942d4fdc8603063a/users/f04d8cddcaa841fa896d21be9dc2a23c/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:30.071211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aaa4d7bd49a4347942d4fdc8603063a, user_id=f04d8cddcaa841fa896d21be9dc2a23c, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:30.072910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:30.163321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1132/2281] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:39:30 2022] PUT /identity/v3/projects/8aaa4d7bd49a4347942d4fdc8603063a/users/f04d8cddcaa841fa896d21be9dc2a23c/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:39:30.173251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2591a-fabc-4216-a749-0d8e431006cb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:30.173546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2591a-fabc-4216-a749-0d8e431006cb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:30.173824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2591a-fabc-4216-a749-0d8e431006cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:30.201796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-74e2591a-fabc-4216-a749-0d8e431006cb None None] MFA Rules not processed for user `f04d8cddcaa841fa896d21be9dc2a23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:39:30.202997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-74e2591a-fabc-4216-a749-0d8e431006cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:30.272094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1150/2282] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:39:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:39:30.283560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:39:30.285351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:30.299196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:30.340774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeb509-0df7-4415-88ab-db40ade96f6d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:39:30.344371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc189f4-82b4-4d9f-abe6-937cc37aa3bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47589bf9e7c4acba200979e3a91a6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:39:30.344999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc189f4-82b4-4d9f-abe6-937cc37aa3bd service glance] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:30.345307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc189f4-82b4-4d9f-abe6-937cc37aa3bd service glance] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:30.345583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc189f4-82b4-4d9f-abe6-937cc37aa3bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:30.353730 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc189f4-82b4-4d9f-abe6-937cc37aa3bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:39:30.355146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc189f4-82b4-4d9f-abe6-937cc37aa3bd service glance] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:39:30.363735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1133/2283] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:39:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:39:40.269157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:40.397204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=20 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:39:51.210661 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b6af-09a8-4b81-b5f8-2ba507c6e531 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:51.212965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b6af-09a8-4b81-b5f8-2ba507c6e531 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:51.212965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b6af-09a8-4b81-b5f8-2ba507c6e531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:51.229476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4bb5b6af-09a8-4b81-b5f8-2ba507c6e531 None None] MFA Rules not processed for user `9d420e4b554e4deab21b89094e0547ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:39:51.230676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4bb5b6af-09a8-4b81-b5f8-2ba507c6e531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:51.284749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1151/2284] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:39:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:39:54.019279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfa1f1-1a5a-472b-b4f0-a29c62fa7c4d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:39:54.019791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfa1f1-1a5a-472b-b4f0-a29c62fa7c4d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:39:54.019887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfa1f1-1a5a-472b-b4f0-a29c62fa7c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:39:54.037570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a4bfa1f1-1a5a-472b-b4f0-a29c62fa7c4d None None] MFA Rules not processed for user `af5fa544bc6440c9830d25e4ad4d5987`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:39:54.038712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a4bfa1f1-1a5a-472b-b4f0-a29c62fa7c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:39:54.101545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1134/2285] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:39:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:40:01.277858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=25 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:04.094485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=25 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:15.342364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5b6af-09a8-4b81-b5f8-2ba507c6e531 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:15.428153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5b6af-09a8-4b81-b5f8-2ba507c6e531 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:15.428998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:15.429984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:15.430276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:15.430607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] PATH_INFO: `/v3/users/f04d8cddcaa841fa896d21be9dc2a23c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:15.432558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04d8cddcaa841fa896d21be9dc2a23c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:15.434571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:15.604953 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04d8cddcaa841fa896d21be9dc2a23c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:15.612220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04d8cddcaa841fa896d21be9dc2a23c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:15.625970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04d8cddcaa841fa896d21be9dc2a23c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:15.632416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04d8cddcaa841fa896d21be9dc2a23c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:15.695314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04d8cddcaa841fa896d21be9dc2a23c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:15.703092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1152/2286] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:40:15 2022] DELETE /identity/v3/users/f04d8cddcaa841fa896d21be9dc2a23c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:40:15.708601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407ae9e-d3fa-4a88-82a4-5dce8d2e84f6 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:15.709088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407ae9e-d3fa-4a88-82a4-5dce8d2e84f6 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:15.709330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407ae9e-d3fa-4a88-82a4-5dce8d2e84f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:15.729722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b407ae9e-d3fa-4a88-82a4-5dce8d2e84f6 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:40:15.731015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b407ae9e-d3fa-4a88-82a4-5dce8d2e84f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:15.798766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1135/2287] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:40:15.808572 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:15.810034 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:15.869746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea9257-1a11-4f39-be70-bbb624c42369 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:15.873490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d0fd3-594f-4ef5-ae97-42825a12ece3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:15.874226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d0fd3-594f-4ef5-ae97-42825a12ece3 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:15.874573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d0fd3-594f-4ef5-ae97-42825a12ece3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:15.874918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d0fd3-594f-4ef5-ae97-42825a12ece3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:15.884252 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d0fd3-594f-4ef5-ae97-42825a12ece3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:15.886389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d0fd3-594f-4ef5-ae97-42825a12ece3 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:15.896239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1153/2288] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:40:16.579187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407ae9e-d3fa-4a88-82a4-5dce8d2e84f6 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:16.580866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b407ae9e-d3fa-4a88-82a4-5dce8d2e84f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:16.603595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407ae9e-d3fa-4a88-82a4-5dce8d2e84f6 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:16.608919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:16.610090 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:16.610375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:16.610652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] PATH_INFO: `/v3/projects/8aaa4d7bd49a4347942d4fdc8603063a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:16.612102 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aaa4d7bd49a4347942d4fdc8603063a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:16.614422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:16.742407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aaa4d7bd49a4347942d4fdc8603063a', 'request_id': 'req-0735062e-ce86-4dcf-8970-94c9389e844e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:16.749948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aaa4d7bd49a4347942d4fdc8603063a', 'request_id': 'req-0735062e-ce86-4dcf-8970-94c9389e844e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:16.757608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aaa4d7bd49a4347942d4fdc8603063a', 'request_id': 'req-0735062e-ce86-4dcf-8970-94c9389e844e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:16.854404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] The token cache is being invalidate because project 8aaa4d7bd49a4347942d4fdc8603063a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:40:16.854888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:16.855610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0735062e-ce86-4dcf-8970-94c9389e844e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:40:16.860673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1136/2289] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:40:16 2022] DELETE /identity/v3/projects/8aaa4d7bd49a4347942d4fdc8603063a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:40:16.862603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1f96b-1acf-4f5f-b781-08b9406b0707 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:16.863112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1f96b-1acf-4f5f-b781-08b9406b0707 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:16.863210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1f96b-1acf-4f5f-b781-08b9406b0707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:16.881626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-89c1f96b-1acf-4f5f-b781-08b9406b0707 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:40:16.883271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-89c1f96b-1acf-4f5f-b781-08b9406b0707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:16.904488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:16.966926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1154/2290] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:40:16.974398 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b830-d09e-4e9f-a47a-a54fa854ae4a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:16.974698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b830-d09e-4e9f-a47a-a54fa854ae4a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:16.974982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b830-d09e-4e9f-a47a-a54fa854ae4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:16.993757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7828b830-d09e-4e9f-a47a-a54fa854ae4a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:40:16.995126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7828b830-d09e-4e9f-a47a-a54fa854ae4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:17.003688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1137/2291] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:40:17.007914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1f96b-1acf-4f5f-b781-08b9406b0707 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:17.018318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1f96b-1acf-4f5f-b781-08b9406b0707 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:17.023290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:17.024027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:17.024389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:17.024732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:17.026176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:40:17.026557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:17.028267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:17.040788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1155/2292] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:40:17.045312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828b830-d09e-4e9f-a47a-a54fa854ae4a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:17.054701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828b830-d09e-4e9f-a47a-a54fa854ae4a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:17.059496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff4f24-61f9-4fac-ac87-7c0834eff430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:17.060141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff4f24-61f9-4fac-ac87-7c0834eff430 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:17.060425 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff4f24-61f9-4fac-ac87-7c0834eff430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:17.060695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff4f24-61f9-4fac-ac87-7c0834eff430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:17.061433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff4f24-61f9-4fac-ac87-7c0834eff430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:17.063334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff4f24-61f9-4fac-ac87-7c0834eff430 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:17.160810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1138/2293] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:40:17 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:40:17.164839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:17.175839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1f93f-359f-4bd4-a7e1-90577b109a59 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:17.239378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed7409-8786-4e61-888e-47d2c3282890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:17.240158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed7409-8786-4e61-888e-47d2c3282890 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:17.240497 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed7409-8786-4e61-888e-47d2c3282890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:17.240839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed7409-8786-4e61-888e-47d2c3282890 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:17.241987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed7409-8786-4e61-888e-47d2c3282890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:17.244520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed7409-8786-4e61-888e-47d2c3282890 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:17.349365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1156/2294] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:40:17 2022] POST /identity/v3/users => generated 282 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:40:17.355334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff4f24-61f9-4fac-ac87-7c0834eff430 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:17.366375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff4f24-61f9-4fac-ac87-7c0834eff430 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:17.371843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dcff5-9de6-4858-87af-b8e58f9bf258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:17.373817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dcff5-9de6-4858-87af-b8e58f9bf258 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:17.373817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dcff5-9de6-4858-87af-b8e58f9bf258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:17.373817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dcff5-9de6-4858-87af-b8e58f9bf258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:17.374397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dcff5-9de6-4858-87af-b8e58f9bf258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:17.375503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dcff5-9de6-4858-87af-b8e58f9bf258 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:17.386942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1139/2295] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:40:17.391029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed7409-8786-4e61-888e-47d2c3282890 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:17.402568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed7409-8786-4e61-888e-47d2c3282890 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:17.407810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916548c6-ae22-4da5-a424-7d391d16fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:17.408612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916548c6-ae22-4da5-a424-7d391d16fd16 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:17.408961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916548c6-ae22-4da5-a424-7d391d16fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:17.409333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916548c6-ae22-4da5-a424-7d391d16fd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:17.410132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916548c6-ae22-4da5-a424-7d391d16fd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:17.411787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916548c6-ae22-4da5-a424-7d391d16fd16 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:17.423378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1157/2296] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:40:17.427792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dcff5-9de6-4858-87af-b8e58f9bf258 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:17.438023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dcff5-9de6-4858-87af-b8e58f9bf258 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:17.442697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:17.443593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:17.443878 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:17.444157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] PATH_INFO: `/v3/projects/bf99a2e0f4f74ffb94e1762f6e917dbd/users/e3c8ac97779b4d1db566cb311cbe6fa6/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:17.460643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf99a2e0f4f74ffb94e1762f6e917dbd, user_id=e3c8ac97779b4d1db566cb311cbe6fa6, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:17.467022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:17.530147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1140/2297] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:40:17 2022] PUT /identity/v3/projects/bf99a2e0f4f74ffb94e1762f6e917dbd/users/e3c8ac97779b4d1db566cb311cbe6fa6/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:40:17.540218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac37ec-e2f0-40d7-9349-1b5e3cafca0e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:17.540618 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac37ec-e2f0-40d7-9349-1b5e3cafca0e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:17.541604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac37ec-e2f0-40d7-9349-1b5e3cafca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:17.577583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-76ac37ec-e2f0-40d7-9349-1b5e3cafca0e None None] MFA Rules not processed for user `e3c8ac97779b4d1db566cb311cbe6fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:40:17.578902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-76ac37ec-e2f0-40d7-9349-1b5e3cafca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:17.652087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1158/2298] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:40:17 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:40:17.663511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:17.665083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:17.706607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:17.753006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ae507-2b02-4f77-b390-feba2033d6af admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:17.757169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63be58-9e3d-4779-bab7-85498ae23bca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:17.758134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63be58-9e3d-4779-bab7-85498ae23bca service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:17.758488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63be58-9e3d-4779-bab7-85498ae23bca service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:17.758774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63be58-9e3d-4779-bab7-85498ae23bca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:17.779273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63be58-9e3d-4779-bab7-85498ae23bca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:17.779273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63be58-9e3d-4779-bab7-85498ae23bca service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:17.783619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1141/2299] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:40:17 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:40:23.284808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acb1d-5418-4628-85d6-a91be663bca3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:23.285245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acb1d-5418-4628-85d6-a91be663bca3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:23.285410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854acb1d-5418-4628-85d6-a91be663bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:23.303328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-854acb1d-5418-4628-85d6-a91be663bca3 None None] MFA Rules not processed for user `e0eb0ab5585747d098bbb0b18770f4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:40:23.304540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-854acb1d-5418-4628-85d6-a91be663bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:23.323266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:23.384016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1159/2300] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:40:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:40:27.843761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=17 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:33.438402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=17 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:44.708721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cad29b-7e89-40d3-b748-462fbc7afafa None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:44.709289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cad29b-7e89-40d3-b748-462fbc7afafa None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:44.709289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cad29b-7e89-40d3-b748-462fbc7afafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:44.729999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-01cad29b-7e89-40d3-b748-462fbc7afafa None None] MFA Rules not processed for user `dc6a3335562448d1934fa18e8f51b119`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:40:44.731241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-01cad29b-7e89-40d3-b748-462fbc7afafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:44.797264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1142/2301] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:40:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:40:46.915834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854acb1d-5418-4628-85d6-a91be663bca3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:46.917314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-854acb1d-5418-4628-85d6-a91be663bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:46.979031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854acb1d-5418-4628-85d6-a91be663bca3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:46.982695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8b2ef-784a-4d17-8848-c3e3ce19b063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:46.983340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8b2ef-784a-4d17-8848-c3e3ce19b063 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:46.983629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8b2ef-784a-4d17-8848-c3e3ce19b063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:46.983908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8b2ef-784a-4d17-8848-c3e3ce19b063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:46.985720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f3b8b2ef-784a-4d17-8848-c3e3ce19b063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:47.056957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8b2ef-784a-4d17-8848-c3e3ce19b063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:47.058440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8b2ef-784a-4d17-8848-c3e3ce19b063 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:47.069351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1160/2302] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:40:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:40:48.388308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cad29b-7e89-40d3-b748-462fbc7afafa None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:40:48.389120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cad29b-7e89-40d3-b748-462fbc7afafa None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:40:48.392728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:40:48.393426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:48.393710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:48.393986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:48.396257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:48.396958 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d48ee-4e15-4e98-af09-70de904aa0f1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:40:48.397436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d48ee-4e15-4e98-af09-70de904aa0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:40:48.397568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d48ee-4e15-4e98-af09-70de904aa0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:40:48.415885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:40:48.418319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8d2d48ee-4e15-4e98-af09-70de904aa0f1 None None] MFA Rules not processed for user `b6709eed72ca46468fd298929ae0f764`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:40:48.418527 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:40:48.419011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8d2d48ee-4e15-4e98-af09-70de904aa0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:40:48.430180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1143/2303] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:40:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:40:48.485367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1161/2304] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:40:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:40:58.425579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=31 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:40:58.483414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=79 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:03.239472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:03.240969 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:03.255045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93ed97-f6a9-48b9-be4f-c2e0ffa1de25 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:03.260651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:03.261310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:03.261598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:03.261877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:03.264253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:03.274708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:03.276293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:03.286118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1144/2305] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:04.359649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d48ee-4e15-4e98-af09-70de904aa0f1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:04.361141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8d2d48ee-4e15-4e98-af09-70de904aa0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:04.378076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d48ee-4e15-4e98-af09-70de904aa0f1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:04.383578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646554d1-76bf-48f6-8fcf-45762ba9f59a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:04.384246 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646554d1-76bf-48f6-8fcf-45762ba9f59a service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:04.384533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646554d1-76bf-48f6-8fcf-45762ba9f59a service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:04.384813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646554d1-76bf-48f6-8fcf-45762ba9f59a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:04.386604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-646554d1-76bf-48f6-8fcf-45762ba9f59a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:04.444111 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646554d1-76bf-48f6-8fcf-45762ba9f59a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:04.445722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646554d1-76bf-48f6-8fcf-45762ba9f59a service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:04.455342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1162/2306] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:05.115241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:05.181514 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ea010-9dbb-4726-8354-190c86f30595 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:05.186763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:05.188362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:05.188362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:05.188362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] PATH_INFO: `/v3/users/e3c8ac97779b4d1db566cb311cbe6fa6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:05.189664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c8ac97779b4d1db566cb311cbe6fa6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:05.192005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:05.410406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c8ac97779b4d1db566cb311cbe6fa6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:05.422630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c8ac97779b4d1db566cb311cbe6fa6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:05.423291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c8ac97779b4d1db566cb311cbe6fa6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:05.423964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c8ac97779b4d1db566cb311cbe6fa6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:05.494472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c8ac97779b4d1db566cb311cbe6fa6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:05.507941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1145/2307] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:41:05 2022] DELETE /identity/v3/users/e3c8ac97779b4d1db566cb311cbe6fa6 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:41:05.515941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2a78-58e3-4192-b47b-e4eaa7b97d77 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:05.516401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2a78-58e3-4192-b47b-e4eaa7b97d77 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:05.516536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e2a78-58e3-4192-b47b-e4eaa7b97d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:05.534209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d91e2a78-58e3-4192-b47b-e4eaa7b97d77 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:05.535418 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d91e2a78-58e3-4192-b47b-e4eaa7b97d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:05.595745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1163/2308] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:05.605560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:05.606978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:05.666464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ddc2b8-0340-4ddc-ab60-5bbbf3435266 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:05.670292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e8836-ca21-44af-8100-bc5ad0b9090a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:05.671141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e8836-ca21-44af-8100-bc5ad0b9090a service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:05.671520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e8836-ca21-44af-8100-bc5ad0b9090a service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:05.671892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e8836-ca21-44af-8100-bc5ad0b9090a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:05.681593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e8836-ca21-44af-8100-bc5ad0b9090a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:05.683171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e8836-ca21-44af-8100-bc5ad0b9090a service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:05.693532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1146/2309] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:41:06.113636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e2a78-58e3-4192-b47b-e4eaa7b97d77 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:06.115527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d91e2a78-58e3-4192-b47b-e4eaa7b97d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:06.128902 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e2a78-58e3-4192-b47b-e4eaa7b97d77 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:06.134461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:06.135761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:06.136122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:06.136502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] PATH_INFO: `/v3/projects/bf99a2e0f4f74ffb94e1762f6e917dbd` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:06.138379 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf99a2e0f4f74ffb94e1762f6e917dbd)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:06.140657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:06.280236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf99a2e0f4f74ffb94e1762f6e917dbd', 'request_id': 'req-f6bab1f9-9e60-4760-a6ee-adff973c8997'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:06.287240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf99a2e0f4f74ffb94e1762f6e917dbd', 'request_id': 'req-f6bab1f9-9e60-4760-a6ee-adff973c8997'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:06.293656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf99a2e0f4f74ffb94e1762f6e917dbd', 'request_id': 'req-f6bab1f9-9e60-4760-a6ee-adff973c8997'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:06.408719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] The token cache is being invalidate because project bf99a2e0f4f74ffb94e1762f6e917dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:41:06.409257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:06.409992 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f6bab1f9-9e60-4760-a6ee-adff973c8997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:06.411649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1164/2310] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:41:06 2022] DELETE /identity/v3/projects/bf99a2e0f4f74ffb94e1762f6e917dbd => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:41:06.416962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4f0a7-d228-4731-81db-9eed52e62595 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:06.417484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4f0a7-d228-4731-81db-9eed52e62595 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:06.417570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4f0a7-d228-4731-81db-9eed52e62595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:06.436088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e9e4f0a7-d228-4731-81db-9eed52e62595 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:06.437391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e9e4f0a7-d228-4731-81db-9eed52e62595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:06.442097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:06.501458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1147/2311] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:06.510349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68ceb-5339-4cfb-9804-3cd57c51b713 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:06.511123 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68ceb-5339-4cfb-9804-3cd57c51b713 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:06.511344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68ceb-5339-4cfb-9804-3cd57c51b713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:06.535223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-84f68ceb-5339-4cfb-9804-3cd57c51b713 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:06.536443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-84f68ceb-5339-4cfb-9804-3cd57c51b713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:06.548325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1165/2312] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:06.549983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4f0a7-d228-4731-81db-9eed52e62595 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:06.563347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4f0a7-d228-4731-81db-9eed52e62595 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:06.566765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:06.567434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:06.567716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:06.567990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:06.576001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:41:06.576327 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:06.578089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:06.596567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1148/2313] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:41:06.600725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f68ceb-5339-4cfb-9804-3cd57c51b713 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:06.621408 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f68ceb-5339-4cfb-9804-3cd57c51b713 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:06.630059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d7548-ec82-4bbc-a665-987f6497977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:06.630738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d7548-ec82-4bbc-a665-987f6497977f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:06.631026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d7548-ec82-4bbc-a665-987f6497977f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:06.631305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d7548-ec82-4bbc-a665-987f6497977f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:06.632032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d7548-ec82-4bbc-a665-987f6497977f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:06.634303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d7548-ec82-4bbc-a665-987f6497977f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:06.650557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:06.751601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1166/2314] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:41:06 2022] POST /identity/v3/projects => generated 389 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:41:06.755850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:06.767001 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f906a6c-5dd1-49b7-afde-c1c32fa8815e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:06.831436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97e70c-09c0-4162-9210-8b1e2c53b7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:06.832132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97e70c-09c0-4162-9210-8b1e2c53b7dd admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:06.832416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97e70c-09c0-4162-9210-8b1e2c53b7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:06.832693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97e70c-09c0-4162-9210-8b1e2c53b7dd admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:06.833431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97e70c-09c0-4162-9210-8b1e2c53b7dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:06.835437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97e70c-09c0-4162-9210-8b1e2c53b7dd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:06.986559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1149/2315] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:41:06 2022] POST /identity/v3/users => generated 295 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:41:06.990963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d7548-ec82-4bbc-a665-987f6497977f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:07.001816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d7548-ec82-4bbc-a665-987f6497977f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:07.007358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa8f4f-b061-4bae-b3c2-2f8e1dd9dbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:07.011639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa8f4f-b061-4bae-b3c2-2f8e1dd9dbfc admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:07.011639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa8f4f-b061-4bae-b3c2-2f8e1dd9dbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:07.011639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa8f4f-b061-4bae-b3c2-2f8e1dd9dbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:07.011639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa8f4f-b061-4bae-b3c2-2f8e1dd9dbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:07.011639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa8f4f-b061-4bae-b3c2-2f8e1dd9dbfc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:07.026761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1167/2316] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:41:07.028162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97e70c-09c0-4162-9210-8b1e2c53b7dd admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:07.039154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97e70c-09c0-4162-9210-8b1e2c53b7dd admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:07.044662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c55a52-a64a-4ff8-896e-f4a8d0618b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:07.045429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c55a52-a64a-4ff8-896e-f4a8d0618b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:07.045725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c55a52-a64a-4ff8-896e-f4a8d0618b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:07.046051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c55a52-a64a-4ff8-896e-f4a8d0618b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:07.046814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c55a52-a64a-4ff8-896e-f4a8d0618b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:07.048585 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c55a52-a64a-4ff8-896e-f4a8d0618b5f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:07.061313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1150/2317] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:41:07.065251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa8f4f-b061-4bae-b3c2-2f8e1dd9dbfc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:07.077333 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa8f4f-b061-4bae-b3c2-2f8e1dd9dbfc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:07.083319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:07.084377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:07.084843 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:07.085301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] PATH_INFO: `/v3/projects/1d817927f6db41bb9e90338e32db9507/users/5cc45cfb39884ff38568368325a28241/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:07.100856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d817927f6db41bb9e90338e32db9507, user_id=5cc45cfb39884ff38568368325a28241, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:07.103554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:07.197728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1168/2318] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:41:07 2022] PUT /identity/v3/projects/1d817927f6db41bb9e90338e32db9507/users/5cc45cfb39884ff38568368325a28241/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:41:07.265735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c3370-b919-4dbc-9571-55f1b4dd3add None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:07.266118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c3370-b919-4dbc-9571-55f1b4dd3add None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:07.266405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c3370-b919-4dbc-9571-55f1b4dd3add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:07.293681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9b3c3370-b919-4dbc-9571-55f1b4dd3add None None] MFA Rules not processed for user `5cc45cfb39884ff38568368325a28241`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:07.294900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9b3c3370-b919-4dbc-9571-55f1b4dd3add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:07.391713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1151/2319] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:07.403516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:07.405261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:07.459741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c3370-b919-4dbc-9571-55f1b4dd3add None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:07.461371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9b3c3370-b919-4dbc-9571-55f1b4dd3add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:07.470066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:07.472839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f142-5eb1-4640-b24e-68e2c34b9d16 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:07.476894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:07.477634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:07.477926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:07.478649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:07.487847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:07.489240 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:07.499297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1169/2320] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:07.534370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c3370-b919-4dbc-9571-55f1b4dd3add None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:07.538464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6447ad-30a8-4f35-bdc5-13506d38551e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:07.539231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6447ad-30a8-4f35-bdc5-13506d38551e service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:07.539526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6447ad-30a8-4f35-bdc5-13506d38551e service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:07.542652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6447ad-30a8-4f35-bdc5-13506d38551e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:07.542652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1e6447ad-30a8-4f35-bdc5-13506d38551e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:07.553064 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6447ad-30a8-4f35-bdc5-13506d38551e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:07.554535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6447ad-30a8-4f35-bdc5-13506d38551e service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:07.563126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1152/2321] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:41:17.494538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:17.663613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=24 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:25.424422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:25.426191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:25.521535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666d375-0e22-48b7-9d06-6a2ee5fbec98 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:25.528648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:25.530422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:25.530871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:25.531306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:25.537510 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:25.558030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:25.559746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:25.587459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1170/2322] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:30.376617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dc596-e04c-4aba-9d8c-49404f077124 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:30.377097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dc596-e04c-4aba-9d8c-49404f077124 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:30.377218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dc596-e04c-4aba-9d8c-49404f077124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:30.394539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-db2dc596-e04c-4aba-9d8c-49404f077124 None None] MFA Rules not processed for user `9d420e4b554e4deab21b89094e0547ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:30.395776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-db2dc596-e04c-4aba-9d8c-49404f077124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:30.454839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1153/2323] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:30.603244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:30.668535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f846e7f-d802-4dba-bc03-474cb3bc8812 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:30.673847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:30.674725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:30.675011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:30.675289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] PATH_INFO: `/v3/users/5cc45cfb39884ff38568368325a28241` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:30.676485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc45cfb39884ff38568368325a28241)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:30.678447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:30.765476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dc596-e04c-4aba-9d8c-49404f077124 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:30.774849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dc596-e04c-4aba-9d8c-49404f077124 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:30.778433 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55508030-daff-4013-b9cd-22baea4d0572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:30.779071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508030-daff-4013-b9cd-22baea4d0572 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:30.779365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508030-daff-4013-b9cd-22baea4d0572 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:30.779644 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55508030-daff-4013-b9cd-22baea4d0572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:30.783822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-55508030-daff-4013-b9cd-22baea4d0572 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:30.794172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55508030-daff-4013-b9cd-22baea4d0572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:30.796442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55508030-daff-4013-b9cd-22baea4d0572 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:30.804520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1154/2324] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:41:30.881541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc45cfb39884ff38568368325a28241'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:30.889897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc45cfb39884ff38568368325a28241'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:30.902345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc45cfb39884ff38568368325a28241'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:30.903193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc45cfb39884ff38568368325a28241'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:31.007933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc45cfb39884ff38568368325a28241'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:31.015503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1171/2325] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:41:30 2022] DELETE /identity/v3/users/5cc45cfb39884ff38568368325a28241 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:41:31.020151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0aae1-a3f9-4096-b377-76dd1dd2a456 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:31.020447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0aae1-a3f9-4096-b377-76dd1dd2a456 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:31.020730 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0aae1-a3f9-4096-b377-76dd1dd2a456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:31.039467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bfc0aae1-a3f9-4096-b377-76dd1dd2a456 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:31.040768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bfc0aae1-a3f9-4096-b377-76dd1dd2a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:31.099049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1155/2326] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:31.111034 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:31.112548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:31.119452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:31.172095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e8e79-f26f-4bc4-abe2-fd53f697ab28 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:31.176016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:31.176649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:31.176927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:31.177234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:31.189386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:31.190851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:31.200200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0aae1-a3f9-4096-b377-76dd1dd2a456 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:31.200932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1172/2327] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:31.201780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bfc0aae1-a3f9-4096-b377-76dd1dd2a456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:31.282614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0aae1-a3f9-4096-b377-76dd1dd2a456 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:31.287679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:31.288356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:31.288635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:31.288909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:31.290737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:31.302285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:31.303864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:31.314284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1156/2328] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:41:31.667231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:31.668715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:31.691954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ad5a-b64b-4315-9d74-bd5577aa1a74 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:31.698229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:31.702700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:31.702700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:31.702700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] PATH_INFO: `/v3/projects/1d817927f6db41bb9e90338e32db9507` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:31.702700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d817927f6db41bb9e90338e32db9507)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:31.704650 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:31.871087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d817927f6db41bb9e90338e32db9507', 'request_id': 'req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:31.877242 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d817927f6db41bb9e90338e32db9507', 'request_id': 'req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:31.887521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d817927f6db41bb9e90338e32db9507', 'request_id': 'req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:32.018459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] The token cache is being invalidate because project 1d817927f6db41bb9e90338e32db9507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:41:32.018758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:32.019407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:32.021013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1173/2329] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:41:31 2022] DELETE /identity/v3/projects/1d817927f6db41bb9e90338e32db9507 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:41:32.281886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:32.287956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:32.303639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2f7bb-ccd7-454a-bc64-3d8a645654ce service placement] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:32.324882 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:32.364681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:32.365591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:32.365881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:32.366164 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] PATH_INFO: `/v3/users/af5fa544bc6440c9830d25e4ad4d5987` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:32.367504 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5fa544bc6440c9830d25e4ad4d5987)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:32.369987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:32.628182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5fa544bc6440c9830d25e4ad4d5987'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:32.639831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5fa544bc6440c9830d25e4ad4d5987'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:32.640486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5fa544bc6440c9830d25e4ad4d5987'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:32.641126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5fa544bc6440c9830d25e4ad4d5987'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:32.731853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-53a4b848-21fb-4835-97d9-a5298ca82961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5fa544bc6440c9830d25e4ad4d5987'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:32.745284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1157/2330] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:41:32 2022] DELETE /identity/v3/users/af5fa544bc6440c9830d25e4ad4d5987 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:41:33.008751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:33.010170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:33.042328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026e2d7-2bb9-461f-b350-ac2b34e0de3d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:33.094957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:33.095833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.096119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.096395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] PATH_INFO: `/v3/projects/e302a2cd63ba46d7a6cef952ddb89242` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.097615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e302a2cd63ba46d7a6cef952ddb89242)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:33.099607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:33.194789 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:33.206165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e302a2cd63ba46d7a6cef952ddb89242', 'request_id': 'req-245bf474-0a49-46d6-8845-0e86bab3bf95'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:33.212448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e302a2cd63ba46d7a6cef952ddb89242', 'request_id': 'req-245bf474-0a49-46d6-8845-0e86bab3bf95'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:33.217921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e302a2cd63ba46d7a6cef952ddb89242', 'request_id': 'req-245bf474-0a49-46d6-8845-0e86bab3bf95'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:33.293423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] The token cache is being invalidate because project e302a2cd63ba46d7a6cef952ddb89242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:41:33.293716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:33.294345 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-245bf474-0a49-46d6-8845-0e86bab3bf95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:41:33.296584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1174/2331] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:41:33 2022] DELETE /identity/v3/projects/e302a2cd63ba46d7a6cef952ddb89242 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:41:33.301750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13045001-f393-4f1e-9790-9d77fee7d480 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.302241 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13045001-f393-4f1e-9790-9d77fee7d480 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.302353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13045001-f393-4f1e-9790-9d77fee7d480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.319788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-13045001-f393-4f1e-9790-9d77fee7d480 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:33.320990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-13045001-f393-4f1e-9790-9d77fee7d480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:33.382653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1158/2332] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:33.390670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae07d7-947c-44b9-b339-ed9ae13d72ae None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.390961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae07d7-947c-44b9-b339-ed9ae13d72ae None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.391236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae07d7-947c-44b9-b339-ed9ae13d72ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.412021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0dae07d7-947c-44b9-b339-ed9ae13d72ae None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:33.413274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0dae07d7-947c-44b9-b339-ed9ae13d72ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:33.423093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1175/2333] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:33.426592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13045001-f393-4f1e-9790-9d77fee7d480 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:33.437347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13045001-f393-4f1e-9790-9d77fee7d480 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:33.446901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:33.447543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.447832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.448110 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.448907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:41:33.449400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:33.451011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:33.463682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1159/2334] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:41:33.467796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae07d7-947c-44b9-b339-ed9ae13d72ae None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:33.476941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae07d7-947c-44b9-b339-ed9ae13d72ae None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:33.481748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef3e31-f2cc-4815-a324-aac339f57317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:33.482391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3e31-f2cc-4815-a324-aac339f57317 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.482676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3e31-f2cc-4815-a324-aac339f57317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.482951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef3e31-f2cc-4815-a324-aac339f57317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.483679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef3e31-f2cc-4815-a324-aac339f57317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:33.485605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef3e31-f2cc-4815-a324-aac339f57317 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:33.590128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1176/2335] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:41:33 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:41:33.594346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:33.604224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6ae8f-9b39-44f3-ad20-325b6f294e0f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:33.631581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:33.662201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23ce38-595a-40c7-8a41-c37c305e2770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:33.662974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23ce38-595a-40c7-8a41-c37c305e2770 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.663259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23ce38-595a-40c7-8a41-c37c305e2770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.663543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23ce38-595a-40c7-8a41-c37c305e2770 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.664306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23ce38-595a-40c7-8a41-c37c305e2770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:33.666267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23ce38-595a-40c7-8a41-c37c305e2770 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:33.792524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1160/2336] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:41:33 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:41:33.794054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef3e31-f2cc-4815-a324-aac339f57317 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:33.805593 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef3e31-f2cc-4815-a324-aac339f57317 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:33.810357 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6a59-5c6c-48d7-8704-659a73d5d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:33.811023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6a59-5c6c-48d7-8704-659a73d5d151 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.811404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6a59-5c6c-48d7-8704-659a73d5d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.812214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6a59-5c6c-48d7-8704-659a73d5d151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.812943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a6a59-5c6c-48d7-8704-659a73d5d151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:33.814707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a6a59-5c6c-48d7-8704-659a73d5d151 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:33.826386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1177/2337] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:41:33.831264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23ce38-595a-40c7-8a41-c37c305e2770 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:33.844264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23ce38-595a-40c7-8a41-c37c305e2770 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:33.849238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c849aa-78e2-4977-a4a9-30b2f7c580fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:33.849898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c849aa-78e2-4977-a4a9-30b2f7c580fd admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.850185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c849aa-78e2-4977-a4a9-30b2f7c580fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.850458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c849aa-78e2-4977-a4a9-30b2f7c580fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.851215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c849aa-78e2-4977-a4a9-30b2f7c580fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:33.852825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c849aa-78e2-4977-a4a9-30b2f7c580fd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:33.863010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1161/2338] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:41:33.867128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6a59-5c6c-48d7-8704-659a73d5d151 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:33.877339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6a59-5c6c-48d7-8704-659a73d5d151 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:33.882108 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:33.882983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:33.883260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:33.883532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] PATH_INFO: `/v3/projects/c0fe831f6e954e05b7d448f1d4f43754/users/2b515cebf18643c38f1cfc2e3fad1db4/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:33.899489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0fe831f6e954e05b7d448f1d4f43754, user_id=2b515cebf18643c38f1cfc2e3fad1db4, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:33.902312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:33.995399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1178/2339] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:41:33 2022] PUT /identity/v3/projects/c0fe831f6e954e05b7d448f1d4f43754/users/2b515cebf18643c38f1cfc2e3fad1db4/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:41:34.000756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0e611-d1c5-4f40-8ff5-b6185d0a3edc None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:34.001048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0e611-d1c5-4f40-8ff5-b6185d0a3edc None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:34.001366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0e611-d1c5-4f40-8ff5-b6185d0a3edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:34.019228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-e7d0e611-d1c5-4f40-8ff5-b6185d0a3edc None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:34.020422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e7d0e611-d1c5-4f40-8ff5-b6185d0a3edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:34.089253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1162/2340] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:34.100415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:34.102266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:34.161609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65a84d-963c-49ca-a3af-729daa0c8d2f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:34.165441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90631511-2005-4d48-847f-c0fa82f41e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:34.166090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90631511-2005-4d48-847f-c0fa82f41e50 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:34.166375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90631511-2005-4d48-847f-c0fa82f41e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:34.166673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90631511-2005-4d48-847f-c0fa82f41e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:34.176022 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90631511-2005-4d48-847f-c0fa82f41e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:34.177491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90631511-2005-4d48-847f-c0fa82f41e50 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:34.187759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1179/2341] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:37.088594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0e611-d1c5-4f40-8ff5-b6185d0a3edc None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:37.090941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e7d0e611-d1c5-4f40-8ff5-b6185d0a3edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:37.156158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0e611-d1c5-4f40-8ff5-b6185d0a3edc None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:37.159810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:37.160453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:37.160732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:37.161003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:37.162836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:37.221150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:37.222797 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:37.233464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1163/2342] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:40.266170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e2a4e-9645-47e1-b792-d4e9c1d43044 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:40.266743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e2a4e-9645-47e1-b792-d4e9c1d43044 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:40.266743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e2a4e-9645-47e1-b792-d4e9c1d43044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:40.299865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-362e2a4e-9645-47e1-b792-d4e9c1d43044 None None] MFA Rules not processed for user `2b515cebf18643c38f1cfc2e3fad1db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:40.303065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-362e2a4e-9645-47e1-b792-d4e9c1d43044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:40.369339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:40.395909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1180/2343] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:40.398766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:40.414832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:40.420543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1701-9c4f-4e6d-a8e0-53232180807f service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:40.441424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:40.442368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:40.442819 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:40.443238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:40.444159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:40.446689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:40.616681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1164/2344] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:41:40 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:41:40.621280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e2a4e-9645-47e1-b792-d4e9c1d43044 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:40.637089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e2a4e-9645-47e1-b792-d4e9c1d43044 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:40.706877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e55801-2dc9-4c7d-953e-d617c3636e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:40.707651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e55801-2dc9-4c7d-953e-d617c3636e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:40.707937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e55801-2dc9-4c7d-953e-d617c3636e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:40.708214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e55801-2dc9-4c7d-953e-d617c3636e32 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:40.708960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e55801-2dc9-4c7d-953e-d617c3636e32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:40.711054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e55801-2dc9-4c7d-953e-d617c3636e32 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:40.759157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:40.762292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:40.844493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1181/2345] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:41:40 2022] POST /identity/v3/users => generated 295 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:41:40.845908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e55801-2dc9-4c7d-953e-d617c3636e32 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:40.846080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74a9e3-3d51-4174-b959-4c9d03dad8ef admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:40.854532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:40.855204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:40.855540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:40.855970 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:40.858493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:40.858654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e55801-2dc9-4c7d-953e-d617c3636e32 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:40.864898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d890e0-2d52-44cd-a494-44c5dcb70e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:40.865424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d890e0-2d52-44cd-a494-44c5dcb70e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:40.867999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d890e0-2d52-44cd-a494-44c5dcb70e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:40.867999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d890e0-2d52-44cd-a494-44c5dcb70e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:40.867999 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d890e0-2d52-44cd-a494-44c5dcb70e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:40.869613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d890e0-2d52-44cd-a494-44c5dcb70e90 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:40.882175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:40.889023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:40.890054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1182/2346] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:41:40.893876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d890e0-2d52-44cd-a494-44c5dcb70e90 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:40.904775 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1165/2347] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:41:40.926780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d890e0-2d52-44cd-a494-44c5dcb70e90 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:40.948583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba40de-b34a-47ee-bf94-0945b2db0bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:40.953530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba40de-b34a-47ee-bf94-0945b2db0bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:40.953819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba40de-b34a-47ee-bf94-0945b2db0bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:40.954103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba40de-b34a-47ee-bf94-0945b2db0bd5 admin admin] PATH_INFO: `/v3/projects/dbf723c339db4b60abd1318b96b6b1fa/users/485a901fccf7445bbc0f27eaeec33ad0/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:40.980243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba40de-b34a-47ee-bf94-0945b2db0bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf723c339db4b60abd1318b96b6b1fa, user_id=485a901fccf7445bbc0f27eaeec33ad0, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:40.983123 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba40de-b34a-47ee-bf94-0945b2db0bd5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:41.061701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1183/2348] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:41:40 2022] PUT /identity/v3/projects/dbf723c339db4b60abd1318b96b6b1fa/users/485a901fccf7445bbc0f27eaeec33ad0/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:41:41.062944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:41.074236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5d094-362a-4a99-9318-4cbc17eebc15 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:41.139084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:41.139758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:41.140041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:41.140318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:41.141039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:41.142978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:41.155900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1166/2349] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:41:41.157012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba40de-b34a-47ee-bf94-0945b2db0bd5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:41.167839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba40de-b34a-47ee-bf94-0945b2db0bd5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:41.173274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd17974-4fe9-4a19-9e61-90721a961fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:41.174306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd17974-4fe9-4a19-9e61-90721a961fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:41.180066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd17974-4fe9-4a19-9e61-90721a961fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:41.180066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd17974-4fe9-4a19-9e61-90721a961fe4 admin admin] PATH_INFO: `/v3/projects/dbf723c339db4b60abd1318b96b6b1fa/users/485a901fccf7445bbc0f27eaeec33ad0/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:41.180066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd17974-4fe9-4a19-9e61-90721a961fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf723c339db4b60abd1318b96b6b1fa, user_id=485a901fccf7445bbc0f27eaeec33ad0, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:41.180066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd17974-4fe9-4a19-9e61-90721a961fe4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:41.192830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1184/2350] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:41:41 2022] PUT /identity/v3/projects/dbf723c339db4b60abd1318b96b6b1fa/users/485a901fccf7445bbc0f27eaeec33ad0/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:41:41.720992 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:41.725093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:41.816561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:41.833466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e88ca9-f91b-4d28-b67d-a1325d7738e0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:41.904238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:41.904918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:41.905263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:41.905542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:41.907389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:41.973445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:41.975159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:41.987370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1167/2351] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:49.159776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895b917-bd22-4d5b-a0b1-995c1241f313 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:49.160344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895b917-bd22-4d5b-a0b1-995c1241f313 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:49.160582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895b917-bd22-4d5b-a0b1-995c1241f313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:49.193290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4895b917-bd22-4d5b-a0b1-995c1241f313 None None] MFA Rules not processed for user `485a901fccf7445bbc0f27eaeec33ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:41:49.195365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4895b917-bd22-4d5b-a0b1-995c1241f313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:41:49.309178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1185/2352] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:41:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:41:49.326498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:41:49.340536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff452d-6097-4a91-af9e-86032f02eada service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:41:49.347351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:41:49.348154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:41:49.348541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:41:49.349229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:41:49.436134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:41:49.438126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:41:49.448838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1168/2353] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:41:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:41:59.304720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:41:59.444078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:01.104073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287e4e1-310c-4d56-94f9-7b7275efd7ea None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:01.104665 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287e4e1-310c-4d56-94f9-7b7275efd7ea None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:01.104766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287e4e1-310c-4d56-94f9-7b7275efd7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:01.128580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0287e4e1-310c-4d56-94f9-7b7275efd7ea None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:01.129987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0287e4e1-310c-4d56-94f9-7b7275efd7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:01.232550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1186/2354] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:42:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:01.363107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:01.364539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:01.434643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba525e34-fb6a-4d86-8a4a-d2bdba01c2d5 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:01.443832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:01.443832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:01.443832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:01.443832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:01.449769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:01.451377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:01.461204 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1169/2355] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:42:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 03:42:03.988888 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287e4e1-310c-4d56-94f9-7b7275efd7ea None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:03.990530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0287e4e1-310c-4d56-94f9-7b7275efd7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:04.052571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287e4e1-310c-4d56-94f9-7b7275efd7ea None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:04.056366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240c9d2-1b30-481a-a295-a5ccd0371e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:04.057604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240c9d2-1b30-481a-a295-a5ccd0371e3d service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:04.057604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240c9d2-1b30-481a-a295-a5ccd0371e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:04.057604 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240c9d2-1b30-481a-a295-a5ccd0371e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:04.059392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6240c9d2-1b30-481a-a295-a5ccd0371e3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:04.130429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240c9d2-1b30-481a-a295-a5ccd0371e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:04.132782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240c9d2-1b30-481a-a295-a5ccd0371e3d service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:04.143560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1187/2356] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:08.848095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:08.849926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:08.943107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52989c9-f1b2-473f-a629-3e6d869bbaa2 service swift] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:08.950342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:08.951181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:08.951569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:08.951950 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:08.954382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:08.967892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:08.970167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:08.982681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1170/2357] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:09.146386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07a67a-326b-4267-aa96-cf74febe5716 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:09.161865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07a67a-326b-4267-aa96-cf74febe5716 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:09.161865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07a67a-326b-4267-aa96-cf74febe5716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:09.164125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9e07a67a-326b-4267-aa96-cf74febe5716 None None] MFA Rules not processed for user `2b515cebf18643c38f1cfc2e3fad1db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:09.165355 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9e07a67a-326b-4267-aa96-cf74febe5716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:09.172167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1188/2358] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:09.221259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:09.222787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:09.239942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1dbde-e81a-4119-b0d2-fd829464444c service placement] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:09.249350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:09.250255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:09.250590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:09.250873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] PATH_INFO: `/v3/users/b6709eed72ca46468fd298929ae0f764` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:09.252259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6709eed72ca46468fd298929ae0f764)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:09.254273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:09.480904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6709eed72ca46468fd298929ae0f764'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:09.498710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6709eed72ca46468fd298929ae0f764'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:09.502720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6709eed72ca46468fd298929ae0f764'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:09.502720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6709eed72ca46468fd298929ae0f764'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:09.537363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07a67a-326b-4267-aa96-cf74febe5716 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:09.548287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07a67a-326b-4267-aa96-cf74febe5716 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:09.552283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:09.553559 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:09.561912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:09.561912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:09.561912 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:09.571169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:09.573249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:09.582803 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1189/2359] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:42:09.583101 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-70d5d978-3caf-4423-a70c-4621f5eb2fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6709eed72ca46468fd298929ae0f764'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:09.585809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:09.601999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1171/2360] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:42:09 2022] DELETE /identity/v3/users/b6709eed72ca46468fd298929ae0f764 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:09.934039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:09.935851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:09.974957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0c0a3-7ee7-4c31-9dac-a878502b7672 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:10.034831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:10.089659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:10.090634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:10.090932 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:10.091287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] PATH_INFO: `/v3/projects/5cab8255c6134aebb9c93493a2342cb1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:10.092512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cab8255c6134aebb9c93493a2342cb1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:10.094615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:10.301933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cab8255c6134aebb9c93493a2342cb1', 'request_id': 'req-dae4edbf-5eb1-4e71-b714-963ba4b0848f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:10.310525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cab8255c6134aebb9c93493a2342cb1', 'request_id': 'req-dae4edbf-5eb1-4e71-b714-963ba4b0848f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:10.316758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cab8255c6134aebb9c93493a2342cb1', 'request_id': 'req-dae4edbf-5eb1-4e71-b714-963ba4b0848f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:10.409226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] The token cache is being invalidate because project 5cab8255c6134aebb9c93493a2342cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:10.409797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:10.414664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dae4edbf-5eb1-4e71-b714-963ba4b0848f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:10.416496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1190/2361] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:09 2022] DELETE /identity/v3/projects/5cab8255c6134aebb9c93493a2342cb1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:10.431346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea11d-82f9-4237-874e-9d697edeb35a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:10.435957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea11d-82f9-4237-874e-9d697edeb35a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:10.435957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea11d-82f9-4237-874e-9d697edeb35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:10.456428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1bdea11d-82f9-4237-874e-9d697edeb35a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:10.459610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1bdea11d-82f9-4237-874e-9d697edeb35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:10.529904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1172/2362] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:10.538253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876a3c-7d3c-4c25-945b-0d27c0b6381c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:10.538978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876a3c-7d3c-4c25-945b-0d27c0b6381c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:10.539335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876a3c-7d3c-4c25-945b-0d27c0b6381c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:10.568696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-97876a3c-7d3c-4c25-945b-0d27c0b6381c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:10.568696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-97876a3c-7d3c-4c25-945b-0d27c0b6381c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:10.570931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1191/2363] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:10.575064 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdea11d-82f9-4237-874e-9d697edeb35a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:10.588690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdea11d-82f9-4237-874e-9d697edeb35a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:10.595810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:10.596727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:10.597216 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:10.597757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:10.598712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:42:10.599196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:10.601067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:10.615762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1173/2364] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:42:10.621780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876a3c-7d3c-4c25-945b-0d27c0b6381c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:10.643125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876a3c-7d3c-4c25-945b-0d27c0b6381c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:10.665917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cff8f1-f8a5-4d26-af13-7b7feaad8679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:10.668401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff8f1-f8a5-4d26-af13-7b7feaad8679 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:10.669409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff8f1-f8a5-4d26-af13-7b7feaad8679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:10.670103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cff8f1-f8a5-4d26-af13-7b7feaad8679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:10.672570 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cff8f1-f8a5-4d26-af13-7b7feaad8679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:10.681795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cff8f1-f8a5-4d26-af13-7b7feaad8679 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:10.803943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1192/2365] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:10 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:10.808325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:10.829972 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6ffe5-d189-4cc4-adf3-b67526cb8e65 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:10.919708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b107944-74a0-4ae6-97ea-56f6ae91a94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:10.920490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107944-74a0-4ae6-97ea-56f6ae91a94b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:10.920855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107944-74a0-4ae6-97ea-56f6ae91a94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:10.921509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b107944-74a0-4ae6-97ea-56f6ae91a94b admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:10.931289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b107944-74a0-4ae6-97ea-56f6ae91a94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:10.933734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b107944-74a0-4ae6-97ea-56f6ae91a94b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:11.033569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1174/2366] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:10 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:42:11.035165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cff8f1-f8a5-4d26-af13-7b7feaad8679 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:11.047219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cff8f1-f8a5-4d26-af13-7b7feaad8679 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:11.055474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53f064-2467-4bfb-b3c6-716d1ff57ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:11.055474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53f064-2467-4bfb-b3c6-716d1ff57ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:11.055474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53f064-2467-4bfb-b3c6-716d1ff57ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:11.055474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53f064-2467-4bfb-b3c6-716d1ff57ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:11.056097 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53f064-2467-4bfb-b3c6-716d1ff57ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:11.057758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53f064-2467-4bfb-b3c6-716d1ff57ac8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:11.074249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1193/2367] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:11.078845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b107944-74a0-4ae6-97ea-56f6ae91a94b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:11.090749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b107944-74a0-4ae6-97ea-56f6ae91a94b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:11.095549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8120a7d-a5e1-418b-8ca2-a6e30d633678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:11.096288 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8120a7d-a5e1-418b-8ca2-a6e30d633678 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:11.096721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8120a7d-a5e1-418b-8ca2-a6e30d633678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:11.097231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8120a7d-a5e1-418b-8ca2-a6e30d633678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:11.097885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8120a7d-a5e1-418b-8ca2-a6e30d633678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:11.099553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8120a7d-a5e1-418b-8ca2-a6e30d633678 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:11.110619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1175/2368] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:42:11.115623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53f064-2467-4bfb-b3c6-716d1ff57ac8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:11.128425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53f064-2467-4bfb-b3c6-716d1ff57ac8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:11.137776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bd06b-e174-4887-a32f-2d529f3cf8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:11.144897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bd06b-e174-4887-a32f-2d529f3cf8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:11.146939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bd06b-e174-4887-a32f-2d529f3cf8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:11.147298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36bd06b-e174-4887-a32f-2d529f3cf8dd admin admin] PATH_INFO: `/v3/projects/31a76a402dd149e8ad108ae9a6d6dc80/users/dc0f69d533834524b9d539e59189ae4e/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:11.170243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bd06b-e174-4887-a32f-2d529f3cf8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a76a402dd149e8ad108ae9a6d6dc80, user_id=dc0f69d533834524b9d539e59189ae4e, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:11.173331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36bd06b-e174-4887-a32f-2d529f3cf8dd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:11.248955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1194/2369] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:11 2022] PUT /identity/v3/projects/31a76a402dd149e8ad108ae9a6d6dc80/users/dc0f69d533834524b9d539e59189ae4e/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:11.260744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6cf47-8f0c-4de0-8a4a-df146a1015d1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:11.261050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6cf47-8f0c-4de0-8a4a-df146a1015d1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:11.261348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6cf47-8f0c-4de0-8a4a-df146a1015d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:11.300118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-daa6cf47-8f0c-4de0-8a4a-df146a1015d1 None None] MFA Rules not processed for user `dc0f69d533834524b9d539e59189ae4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:11.301603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-daa6cf47-8f0c-4de0-8a4a-df146a1015d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:11.375831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:11.419217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1176/2370] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:11.423504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bd06b-e174-4887-a32f-2d529f3cf8dd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:11.484115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:11.496879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36bd06b-e174-4887-a32f-2d529f3cf8dd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:11.503020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c4435-7163-45e5-bbe0-f01fc3595bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:11.503926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c4435-7163-45e5-bbe0-f01fc3595bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:11.504207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c4435-7163-45e5-bbe0-f01fc3595bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:11.504482 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c4435-7163-45e5-bbe0-f01fc3595bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:11.505373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c4435-7163-45e5-bbe0-f01fc3595bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:11.514882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c4435-7163-45e5-bbe0-f01fc3595bb0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:11.631084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1195/2371] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:11 2022] POST /identity/v3/projects => generated 381 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:11.635319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6cf47-8f0c-4de0-8a4a-df146a1015d1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:11.647191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6cf47-8f0c-4de0-8a4a-df146a1015d1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:11.721231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa44ea-8fff-4597-b846-d2c74dcddfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:11.724416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa44ea-8fff-4597-b846-d2c74dcddfec admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:11.724416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa44ea-8fff-4597-b846-d2c74dcddfec admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:11.724416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa44ea-8fff-4597-b846-d2c74dcddfec admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:11.724416 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa44ea-8fff-4597-b846-d2c74dcddfec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:11.725224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa44ea-8fff-4597-b846-d2c74dcddfec admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:11.836756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1177/2372] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:11 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:42:11.841549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c4435-7163-45e5-bbe0-f01fc3595bb0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:11.855445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c4435-7163-45e5-bbe0-f01fc3595bb0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:11.862588 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71156b6b-5295-4e4e-bdf8-6bbd9d0c2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:11.867615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71156b6b-5295-4e4e-bdf8-6bbd9d0c2ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:11.867615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71156b6b-5295-4e4e-bdf8-6bbd9d0c2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:11.867615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71156b6b-5295-4e4e-bdf8-6bbd9d0c2ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:11.867615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71156b6b-5295-4e4e-bdf8-6bbd9d0c2ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:11.867615 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71156b6b-5295-4e4e-bdf8-6bbd9d0c2ed5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:11.880840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1196/2373] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:11.885125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa44ea-8fff-4597-b846-d2c74dcddfec admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:11.895718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa44ea-8fff-4597-b846-d2c74dcddfec admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:11.901658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3418d-665e-4fae-91c6-7de96a93af35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:11.902646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3418d-665e-4fae-91c6-7de96a93af35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:11.903057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3418d-665e-4fae-91c6-7de96a93af35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:11.903411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3418d-665e-4fae-91c6-7de96a93af35 admin admin] PATH_INFO: `/v3/projects/34a9dc521ac147338a8634fa17b67321/users/b7516f350f3040388dcb85e4900ef489/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:11.921215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3418d-665e-4fae-91c6-7de96a93af35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a9dc521ac147338a8634fa17b67321, user_id=b7516f350f3040388dcb85e4900ef489, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:11.924299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3418d-665e-4fae-91c6-7de96a93af35 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:12.017217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1178/2374] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:11 2022] PUT /identity/v3/projects/34a9dc521ac147338a8634fa17b67321/users/b7516f350f3040388dcb85e4900ef489/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:12.021042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71156b6b-5295-4e4e-bdf8-6bbd9d0c2ed5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:12.043781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71156b6b-5295-4e4e-bdf8-6bbd9d0c2ed5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:12.123204 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d5ca9-d741-43f4-8aaa-f581374e8b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:12.123896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5ca9-d741-43f4-8aaa-f581374e8b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:12.124186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5ca9-d741-43f4-8aaa-f581374e8b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:12.124467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d5ca9-d741-43f4-8aaa-f581374e8b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:12.125277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5ca9-d741-43f4-8aaa-f581374e8b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:12.127004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d5ca9-d741-43f4-8aaa-f581374e8b40 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:12.141382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1197/2375] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:12.144341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3418d-665e-4fae-91c6-7de96a93af35 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:12.160299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3418d-665e-4fae-91c6-7de96a93af35 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:12.168054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:12.170581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:12.170869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:12.171152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] PATH_INFO: `/v3/projects/34a9dc521ac147338a8634fa17b67321/users/b7516f350f3040388dcb85e4900ef489/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:12.173597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34a9dc521ac147338a8634fa17b67321, user_id=b7516f350f3040388dcb85e4900ef489, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:12.177280 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:12.188574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1179/2376] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:12 2022] PUT /identity/v3/projects/34a9dc521ac147338a8634fa17b67321/users/b7516f350f3040388dcb85e4900ef489/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:12.201663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76994f2-e85e-4841-bafd-191eefec6a5c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:12.202257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76994f2-e85e-4841-bafd-191eefec6a5c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:12.202601 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76994f2-e85e-4841-bafd-191eefec6a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:12.236432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d76994f2-e85e-4841-bafd-191eefec6a5c None None] MFA Rules not processed for user `b7516f350f3040388dcb85e4900ef489`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:12.238107 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d76994f2-e85e-4841-bafd-191eefec6a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:12.322312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1198/2377] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:12.334565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:12.336375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:12.409545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43caf085-9029-456c-850b-9a0265b8ed68 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:12.413494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53962e-e7a7-4933-8bb5-f212098a893e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:12.414195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53962e-e7a7-4933-8bb5-f212098a893e service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:12.414476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53962e-e7a7-4933-8bb5-f212098a893e service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:12.414754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53962e-e7a7-4933-8bb5-f212098a893e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:12.425039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53962e-e7a7-4933-8bb5-f212098a893e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:12.426804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53962e-e7a7-4933-8bb5-f212098a893e service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:12.438802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1180/2378] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:13.542199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76994f2-e85e-4841-bafd-191eefec6a5c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:13.551344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76994f2-e85e-4841-bafd-191eefec6a5c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:13.556589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7516f350f3040388dcb85e4900ef489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34a9dc521ac147338a8634fa17b67321', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:13.557269 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:13.557556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:13.557834 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:13.558556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:42:13.558862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:13.560441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:13.572686 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1199/2379] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:42:13.589213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53962e-e7a7-4933-8bb5-f212098a893e service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:13.589213 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53962e-e7a7-4933-8bb5-f212098a893e service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:13.593579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810e66-8333-42ac-8cef-41b1d0e38b51 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7516f350f3040388dcb85e4900ef489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34a9dc521ac147338a8634fa17b67321', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:13.594297 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810e66-8333-42ac-8cef-41b1d0e38b51 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:13.594712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810e66-8333-42ac-8cef-41b1d0e38b51 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:13.595194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8810e66-8333-42ac-8cef-41b1d0e38b51 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:13.595942 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8810e66-8333-42ac-8cef-41b1d0e38b51 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:13.597930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8810e66-8333-42ac-8cef-41b1d0e38b51 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:13.707615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1181/2380] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:13 2022] POST /identity/v3/projects => generated 353 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:13.711776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:13.722187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3bc3a-72d4-49bd-ad0f-3dca533f8e18 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:13.758350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:13.784215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7516f350f3040388dcb85e4900ef489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34a9dc521ac147338a8634fa17b67321', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:13.788827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:13.788827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:13.788827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:13.788827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:42:13.788827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:13.789592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:13.800701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1200/2381] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:42:13.837283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810e66-8333-42ac-8cef-41b1d0e38b51 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:13.850459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8810e66-8333-42ac-8cef-41b1d0e38b51 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:13.857153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7516f350f3040388dcb85e4900ef489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34a9dc521ac147338a8634fa17b67321', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:13.858407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:13.858834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:13.859178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] PATH_INFO: `/v3/projects/df97a7e6bfba49808372876c07d5f4c8` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:13.860519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Authorizing `identity:delete_project(project_id=df97a7e6bfba49808372876c07d5f4c8)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:13.866005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:13.894142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:14.014434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df97a7e6bfba49808372876c07d5f4c8', 'request_id': 'req-baeab9cb-2e5a-4865-a382-95ab8843a317'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:14.020663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df97a7e6bfba49808372876c07d5f4c8', 'request_id': 'req-baeab9cb-2e5a-4865-a382-95ab8843a317'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:14.026871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df97a7e6bfba49808372876c07d5f4c8', 'request_id': 'req-baeab9cb-2e5a-4865-a382-95ab8843a317'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:14.113630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] The token cache is being invalidate because project df97a7e6bfba49808372876c07d5f4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:14.113929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:14.114579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-baeab9cb-2e5a-4865-a382-95ab8843a317 tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:14.116178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1182/2382] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:13 2022] DELETE /identity/v3/projects/df97a7e6bfba49808372876c07d5f4c8 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:14.806328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:14.809329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:14.907377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875e7db-d1c8-4cbe-baac-de76f58d855c tempest-VolumeQuotaClassesTest-1955931879 tempest-VolumeQuotaClassesTest-1955931879-project-admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:15.008088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:15.008857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:15.009170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:15.009568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] PATH_INFO: `/v3/users/dc0f69d533834524b9d539e59189ae4e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:15.011459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0f69d533834524b9d539e59189ae4e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:15.013860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:15.227871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0f69d533834524b9d539e59189ae4e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:15.235736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0f69d533834524b9d539e59189ae4e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:15.252859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0f69d533834524b9d539e59189ae4e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:15.253810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0f69d533834524b9d539e59189ae4e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:15.259620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:15.323276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0f69d533834524b9d539e59189ae4e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:15.332462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1201/2383] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:42:14 2022] DELETE /identity/v3/users/dc0f69d533834524b9d539e59189ae4e => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:15.334409 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b546d85-4e8e-4d19-a044-7fa56fd68b18 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:15.334956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b546d85-4e8e-4d19-a044-7fa56fd68b18 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:15.335298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b546d85-4e8e-4d19-a044-7fa56fd68b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:15.355903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7b546d85-4e8e-4d19-a044-7fa56fd68b18 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:15.357221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b546d85-4e8e-4d19-a044-7fa56fd68b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:15.420858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1183/2384] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:42:15.431170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:15.432633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:15.506994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf363c-add7-424a-9ebf-0b187354f8b0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:15.512297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6febed2-58bf-4a48-afbc-a88fe9b5c84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:15.512974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6febed2-58bf-4a48-afbc-a88fe9b5c84e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:15.513459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6febed2-58bf-4a48-afbc-a88fe9b5c84e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:15.513744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6febed2-58bf-4a48-afbc-a88fe9b5c84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:15.526081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6febed2-58bf-4a48-afbc-a88fe9b5c84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:15.531002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6febed2-58bf-4a48-afbc-a88fe9b5c84e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:15.554707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1202/2385] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:16.000394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b546d85-4e8e-4d19-a044-7fa56fd68b18 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:16.002575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b546d85-4e8e-4d19-a044-7fa56fd68b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:16.015369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b546d85-4e8e-4d19-a044-7fa56fd68b18 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:16.020224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:16.021096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:16.021549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:16.022225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] PATH_INFO: `/v3/projects/31a76a402dd149e8ad108ae9a6d6dc80` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:16.023349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a76a402dd149e8ad108ae9a6d6dc80)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:16.025326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:16.193700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a76a402dd149e8ad108ae9a6d6dc80', 'request_id': 'req-48b97fec-20d9-41a8-81f4-65f91e22840c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.200496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a76a402dd149e8ad108ae9a6d6dc80', 'request_id': 'req-48b97fec-20d9-41a8-81f4-65f91e22840c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.207148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a76a402dd149e8ad108ae9a6d6dc80', 'request_id': 'req-48b97fec-20d9-41a8-81f4-65f91e22840c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.299920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] The token cache is being invalidate because project 31a76a402dd149e8ad108ae9a6d6dc80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:16.300210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.300929 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.302513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1184/2386] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:15 2022] DELETE /identity/v3/projects/31a76a402dd149e8ad108ae9a6d6dc80 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:16.306617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6febed2-58bf-4a48-afbc-a88fe9b5c84e service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:16.316265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6febed2-58bf-4a48-afbc-a88fe9b5c84e service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:16.372063 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:16.372980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:16.373298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:16.373588 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] PATH_INFO: `/v3/users/b7516f350f3040388dcb85e4900ef489` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:16.374767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7516f350f3040388dcb85e4900ef489)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:16.376686 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:16.535322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7516f350f3040388dcb85e4900ef489'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.543462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7516f350f3040388dcb85e4900ef489'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.559882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7516f350f3040388dcb85e4900ef489'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.560786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7516f350f3040388dcb85e4900ef489'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.622954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-bc3f6db6-6e38-4376-994b-f9eeb4bb6b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7516f350f3040388dcb85e4900ef489'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:16.629844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1203/2387] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:42:16 2022] DELETE /identity/v3/users/b7516f350f3040388dcb85e4900ef489 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:17.142791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:17.148768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:17.163698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b97fec-20d9-41a8-81f4-65f91e22840c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:17.206661 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:17.265450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:17.266417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:17.266772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:17.267119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] PATH_INFO: `/v3/projects/34a9dc521ac147338a8634fa17b67321` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:17.268405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34a9dc521ac147338a8634fa17b67321)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:17.270435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:17.401296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34a9dc521ac147338a8634fa17b67321', 'request_id': 'req-af7259cd-9fed-4f59-a958-9879abaac8b0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:17.409334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34a9dc521ac147338a8634fa17b67321', 'request_id': 'req-af7259cd-9fed-4f59-a958-9879abaac8b0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:17.414029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34a9dc521ac147338a8634fa17b67321', 'request_id': 'req-af7259cd-9fed-4f59-a958-9879abaac8b0'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:17.529319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] The token cache is being invalidate because project 34a9dc521ac147338a8634fa17b67321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:17.529319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:17.530243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-af7259cd-9fed-4f59-a958-9879abaac8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:17.532592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1185/2388] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:17 2022] DELETE /identity/v3/projects/34a9dc521ac147338a8634fa17b67321 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:17.538174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1381cdf-3dab-4ac5-8757-2e7e09ee0f81 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:17.538704 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1381cdf-3dab-4ac5-8757-2e7e09ee0f81 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:17.538859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1381cdf-3dab-4ac5-8757-2e7e09ee0f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:17.559102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e1381cdf-3dab-4ac5-8757-2e7e09ee0f81 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:17.560346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e1381cdf-3dab-4ac5-8757-2e7e09ee0f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:17.574968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:17.624955 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1204/2389] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:17.633290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b329-5f27-4177-b7bc-06972f51a991 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:17.633593 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b329-5f27-4177-b7bc-06972f51a991 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:17.633889 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b329-5f27-4177-b7bc-06972f51a991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:17.652937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6f80b329-5f27-4177-b7bc-06972f51a991 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:17.654202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6f80b329-5f27-4177-b7bc-06972f51a991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:17.662312 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1186/2390] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:17.666526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1381cdf-3dab-4ac5-8757-2e7e09ee0f81 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:17.676473 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1381cdf-3dab-4ac5-8757-2e7e09ee0f81 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:17.682175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:17.682836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:17.683119 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:17.683395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:17.684122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:42:17.684431 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:17.686075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:17.699229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1205/2391] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:42:17.703660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80b329-5f27-4177-b7bc-06972f51a991 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:17.713220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80b329-5f27-4177-b7bc-06972f51a991 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:17.720123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8266d-8826-4255-8397-69c3b2055eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:17.720123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8266d-8826-4255-8397-69c3b2055eac admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:17.720123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8266d-8826-4255-8397-69c3b2055eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:17.720123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8266d-8826-4255-8397-69c3b2055eac admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:17.720625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8266d-8826-4255-8397-69c3b2055eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:17.721829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d8266d-8826-4255-8397-69c3b2055eac admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:17.801763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1187/2392] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:17 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:42:17.803507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:17.813522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4769d35-13a1-49bb-ba30-aed8af29ab5f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:17.870639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9809d5-2ba2-4484-b7fb-e0f345917a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:17.871346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9809d5-2ba2-4484-b7fb-e0f345917a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:17.871633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9809d5-2ba2-4484-b7fb-e0f345917a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:17.871915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9809d5-2ba2-4484-b7fb-e0f345917a8b admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:17.872607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9809d5-2ba2-4484-b7fb-e0f345917a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:17.874569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9809d5-2ba2-4484-b7fb-e0f345917a8b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:18.007763 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1206/2393] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:17 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:42:18.011473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8266d-8826-4255-8397-69c3b2055eac admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:18.021716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8266d-8826-4255-8397-69c3b2055eac admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:18.026171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f418b0-ff77-4748-ab67-222533b09cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:18.026817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f418b0-ff77-4748-ab67-222533b09cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.027879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f418b0-ff77-4748-ab67-222533b09cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.027879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f418b0-ff77-4748-ab67-222533b09cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.028071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f418b0-ff77-4748-ab67-222533b09cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:18.029693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f418b0-ff77-4748-ab67-222533b09cd0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:18.040228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1188/2394] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:18.044131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9809d5-2ba2-4484-b7fb-e0f345917a8b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:18.054988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9809d5-2ba2-4484-b7fb-e0f345917a8b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:18.059880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48250a3a-3298-49ed-a283-16c3d57de2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:18.060543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250a3a-3298-49ed-a283-16c3d57de2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.060831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250a3a-3298-49ed-a283-16c3d57de2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.061125 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250a3a-3298-49ed-a283-16c3d57de2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.061867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48250a3a-3298-49ed-a283-16c3d57de2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:18.063456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48250a3a-3298-49ed-a283-16c3d57de2b3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:18.078328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1207/2395] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:18.079125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f418b0-ff77-4748-ab67-222533b09cd0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:18.094424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f418b0-ff77-4748-ab67-222533b09cd0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:18.095657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687727f1-0cac-4a4b-915f-a17e500ee38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:18.096546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687727f1-0cac-4a4b-915f-a17e500ee38c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.096868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687727f1-0cac-4a4b-915f-a17e500ee38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.097098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687727f1-0cac-4a4b-915f-a17e500ee38c admin admin] PATH_INFO: `/v3/projects/ff7fdf06df8942baae5674293f41788c/users/a50b96649c9941d2ac45e9e02c3d0063/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.116592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687727f1-0cac-4a4b-915f-a17e500ee38c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7fdf06df8942baae5674293f41788c, user_id=a50b96649c9941d2ac45e9e02c3d0063, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:18.120322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687727f1-0cac-4a4b-915f-a17e500ee38c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:18.200619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1189/2396] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:18 2022] PUT /identity/v3/projects/ff7fdf06df8942baae5674293f41788c/users/a50b96649c9941d2ac45e9e02c3d0063/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:18.212251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e4b8-d96e-42da-b14b-29ccf39142b0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.212591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e4b8-d96e-42da-b14b-29ccf39142b0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.212885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e4b8-d96e-42da-b14b-29ccf39142b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.248255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9ae3e4b8-d96e-42da-b14b-29ccf39142b0 None None] MFA Rules not processed for user `a50b96649c9941d2ac45e9e02c3d0063`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:18.249721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9ae3e4b8-d96e-42da-b14b-29ccf39142b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:18.319608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1208/2397] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:18.324105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687727f1-0cac-4a4b-915f-a17e500ee38c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:18.384739 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:18.395041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687727f1-0cac-4a4b-915f-a17e500ee38c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:18.399794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092c227-f0af-4d89-a3cb-f1b5eb27210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:18.400455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092c227-f0af-4d89-a3cb-f1b5eb27210e admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.400739 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092c227-f0af-4d89-a3cb-f1b5eb27210e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.401015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092c227-f0af-4d89-a3cb-f1b5eb27210e admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.401753 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092c227-f0af-4d89-a3cb-f1b5eb27210e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:18.403808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092c227-f0af-4d89-a3cb-f1b5eb27210e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:18.611131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1190/2398] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:18 2022] POST /identity/v3/projects => generated 379 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:18.615186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3e4b8-d96e-42da-b14b-29ccf39142b0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:18.626404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3e4b8-d96e-42da-b14b-29ccf39142b0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:18.636446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:18.693196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154d9ae-0863-4620-8dbe-6dc5c4805a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:18.693880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154d9ae-0863-4620-8dbe-6dc5c4805a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.694258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154d9ae-0863-4620-8dbe-6dc5c4805a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.694446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154d9ae-0863-4620-8dbe-6dc5c4805a45 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.695213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154d9ae-0863-4620-8dbe-6dc5c4805a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:18.697059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154d9ae-0863-4620-8dbe-6dc5c4805a45 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:18.826303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1209/2399] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:18 2022] POST /identity/v3/users => generated 290 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:18.830208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092c227-f0af-4d89-a3cb-f1b5eb27210e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:18.842471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092c227-f0af-4d89-a3cb-f1b5eb27210e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:18.847320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a94d-ed7b-4769-9af4-e2a82eccd842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:18.848036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a94d-ed7b-4769-9af4-e2a82eccd842 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.848321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a94d-ed7b-4769-9af4-e2a82eccd842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.848609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a94d-ed7b-4769-9af4-e2a82eccd842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.849417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514a94d-ed7b-4769-9af4-e2a82eccd842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:18.852523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514a94d-ed7b-4769-9af4-e2a82eccd842 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:18.864279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1191/2400] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:18.868787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154d9ae-0863-4620-8dbe-6dc5c4805a45 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:18.879597 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154d9ae-0863-4620-8dbe-6dc5c4805a45 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:18.884827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3ac65-860a-42e2-bdb6-a8ab81c703da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:18.885552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3ac65-860a-42e2-bdb6-a8ab81c703da admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.885950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3ac65-860a-42e2-bdb6-a8ab81c703da admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.886250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3ac65-860a-42e2-bdb6-a8ab81c703da admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.887069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3ac65-860a-42e2-bdb6-a8ab81c703da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:18.888641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3ac65-860a-42e2-bdb6-a8ab81c703da admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:18.903014 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1210/2401] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:18.908595 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a94d-ed7b-4769-9af4-e2a82eccd842 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:18.921881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a94d-ed7b-4769-9af4-e2a82eccd842 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:18.926656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3860a-0a57-4683-a5f2-4c1457f84b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:18.928023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3860a-0a57-4683-a5f2-4c1457f84b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:18.928372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3860a-0a57-4683-a5f2-4c1457f84b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:18.928717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3860a-0a57-4683-a5f2-4c1457f84b9c admin admin] PATH_INFO: `/v3/projects/13683de02d4c440fbdc1acd44fece9a0/users/4e7e7b9e51a74b1989bce1f983044602/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:18.946522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3860a-0a57-4683-a5f2-4c1457f84b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=13683de02d4c440fbdc1acd44fece9a0, user_id=4e7e7b9e51a74b1989bce1f983044602, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:18.949371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3860a-0a57-4683-a5f2-4c1457f84b9c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:19.063330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1192/2402] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:18 2022] PUT /identity/v3/projects/13683de02d4c440fbdc1acd44fece9a0/users/4e7e7b9e51a74b1989bce1f983044602/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:19.074147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22443bc5-cc5a-4764-9fd4-f1e0f366d89d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:19.074448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22443bc5-cc5a-4764-9fd4-f1e0f366d89d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:19.074767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22443bc5-cc5a-4764-9fd4-f1e0f366d89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:19.103086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-22443bc5-cc5a-4764-9fd4-f1e0f366d89d None None] MFA Rules not processed for user `4e7e7b9e51a74b1989bce1f983044602`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:19.104393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-22443bc5-cc5a-4764-9fd4-f1e0f366d89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:19.189775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1211/2403] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:42:19.195307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3860a-0a57-4683-a5f2-4c1457f84b9c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:19.281128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3860a-0a57-4683-a5f2-4c1457f84b9c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:19.298818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93769112-e404-4fac-b586-15fe9b2b87b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:19.298818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93769112-e404-4fac-b586-15fe9b2b87b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:19.298818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93769112-e404-4fac-b586-15fe9b2b87b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:19.298818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93769112-e404-4fac-b586-15fe9b2b87b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:19.299538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93769112-e404-4fac-b586-15fe9b2b87b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:19.299538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93769112-e404-4fac-b586-15fe9b2b87b8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:19.382397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1193/2404] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:19 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:19.383573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22443bc5-cc5a-4764-9fd4-f1e0f366d89d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:19.394500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22443bc5-cc5a-4764-9fd4-f1e0f366d89d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:19.454300 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f440fd0-a875-4f5c-8b4c-ae229c24a758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:19.454978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f440fd0-a875-4f5c-8b4c-ae229c24a758 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:19.455254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f440fd0-a875-4f5c-8b4c-ae229c24a758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:19.455529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f440fd0-a875-4f5c-8b4c-ae229c24a758 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:19.456208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f440fd0-a875-4f5c-8b4c-ae229c24a758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:19.458144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f440fd0-a875-4f5c-8b4c-ae229c24a758 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:19.590261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1212/2405] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:19 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:42:19.594873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93769112-e404-4fac-b586-15fe9b2b87b8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:19.607687 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93769112-e404-4fac-b586-15fe9b2b87b8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:19.615601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9a239-5fe3-4af8-b5e0-4c413b977c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:19.615601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9a239-5fe3-4af8-b5e0-4c413b977c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:19.615601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9a239-5fe3-4af8-b5e0-4c413b977c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:19.615601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9a239-5fe3-4af8-b5e0-4c413b977c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:19.616024 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9a239-5fe3-4af8-b5e0-4c413b977c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:19.616956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9a239-5fe3-4af8-b5e0-4c413b977c73 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:19.628613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1194/2406] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:19.632695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f440fd0-a875-4f5c-8b4c-ae229c24a758 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:19.642924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f440fd0-a875-4f5c-8b4c-ae229c24a758 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:19.647522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00441a5a-8ca1-435a-8627-e575022c0858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:19.648386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00441a5a-8ca1-435a-8627-e575022c0858 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:19.648666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00441a5a-8ca1-435a-8627-e575022c0858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:19.648939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00441a5a-8ca1-435a-8627-e575022c0858 admin admin] PATH_INFO: `/v3/projects/e66e3943aebf45cea7afb5f36a04d6a0/users/d88ff7656d4e4c719fe7e1b786f24b52/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:19.657460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:19.664376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00441a5a-8ca1-435a-8627-e575022c0858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66e3943aebf45cea7afb5f36a04d6a0, user_id=d88ff7656d4e4c719fe7e1b786f24b52, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:19.667024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00441a5a-8ca1-435a-8627-e575022c0858 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:19.754667 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1213/2407] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:19 2022] PUT /identity/v3/projects/e66e3943aebf45cea7afb5f36a04d6a0/users/d88ff7656d4e4c719fe7e1b786f24b52/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:19.755835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9a239-5fe3-4af8-b5e0-4c413b977c73 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:19.768910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9a239-5fe3-4af8-b5e0-4c413b977c73 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:19.832362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e79f4e-031d-4708-be93-dcd3a656e39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:19.833032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79f4e-031d-4708-be93-dcd3a656e39d admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:19.833341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79f4e-031d-4708-be93-dcd3a656e39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:19.833647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e79f4e-031d-4708-be93-dcd3a656e39d admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:19.834364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e79f4e-031d-4708-be93-dcd3a656e39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:19.836012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e79f4e-031d-4708-be93-dcd3a656e39d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:19.848000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1195/2408] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:19.860451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00441a5a-8ca1-435a-8627-e575022c0858 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:19.862049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00441a5a-8ca1-435a-8627-e575022c0858 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:19.867585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:19.868721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:19.869007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:19.869322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] PATH_INFO: `/v3/projects/e66e3943aebf45cea7afb5f36a04d6a0/users/d88ff7656d4e4c719fe7e1b786f24b52/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:19.871318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e66e3943aebf45cea7afb5f36a04d6a0, user_id=d88ff7656d4e4c719fe7e1b786f24b52, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:19.875620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:19.884478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1214/2409] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:19 2022] PUT /identity/v3/projects/e66e3943aebf45cea7afb5f36a04d6a0/users/d88ff7656d4e4c719fe7e1b786f24b52/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:19.894822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4f70-c924-4e03-adc6-e15441140b52 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:19.895115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4f70-c924-4e03-adc6-e15441140b52 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:19.895400 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca4f70-c924-4e03-adc6-e15441140b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:19.927038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:19.930757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-90ca4f70-c924-4e03-adc6-e15441140b52 None None] MFA Rules not processed for user `d88ff7656d4e4c719fe7e1b786f24b52`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:19.931957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-90ca4f70-c924-4e03-adc6-e15441140b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:19.999429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1196/2410] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:20.010927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:20.012769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:20.079632 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ffcd8-13a2-4e5a-837c-34cc95c4abd9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:20.083329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c266a-3474-4912-9022-c59f06e06916 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:20.084502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c266a-3474-4912-9022-c59f06e06916 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:20.084783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c266a-3474-4912-9022-c59f06e06916 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:20.085055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c266a-3474-4912-9022-c59f06e06916 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:20.094287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c266a-3474-4912-9022-c59f06e06916 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:20.096291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c266a-3474-4912-9022-c59f06e06916 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:20.106561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1215/2411] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:20.225603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4f70-c924-4e03-adc6-e15441140b52 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:20.235079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca4f70-c924-4e03-adc6-e15441140b52 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:20.238695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63ef35-5d7f-4e33-a28e-15418a6b12bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:20.239331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ef35-5d7f-4e33-a28e-15418a6b12bc service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:20.239613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ef35-5d7f-4e33-a28e-15418a6b12bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:20.239955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ef35-5d7f-4e33-a28e-15418a6b12bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:20.297766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63ef35-5d7f-4e33-a28e-15418a6b12bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:20.299223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63ef35-5d7f-4e33-a28e-15418a6b12bc service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:20.309834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1197/2412] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:24.428771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c266a-3474-4912-9022-c59f06e06916 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:24.496163 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c266a-3474-4912-9022-c59f06e06916 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:24.502454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:24.503506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:24.503858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:24.504380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] PATH_INFO: `/v3/users/a50b96649c9941d2ac45e9e02c3d0063` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:24.506225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50b96649c9941d2ac45e9e02c3d0063)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:24.508364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:24.744568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50b96649c9941d2ac45e9e02c3d0063'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:24.753517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50b96649c9941d2ac45e9e02c3d0063'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:24.766708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50b96649c9941d2ac45e9e02c3d0063'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:24.767462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50b96649c9941d2ac45e9e02c3d0063'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:24.771533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:24.885766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50b96649c9941d2ac45e9e02c3d0063'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:24.894984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1216/2413] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:42:24 2022] DELETE /identity/v3/users/a50b96649c9941d2ac45e9e02c3d0063 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:24.898977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419c95-fcca-4177-be82-d3277110ecde None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:24.899497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419c95-fcca-4177-be82-d3277110ecde None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:24.899597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d419c95-fcca-4177-be82-d3277110ecde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:24.932619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-7d419c95-fcca-4177-be82-d3277110ecde None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:24.934205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7d419c95-fcca-4177-be82-d3277110ecde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:25.028472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1198/2414] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:25.036278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:25.037985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:25.113359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f67082-b948-4c30-9759-c672a53c563a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:25.117214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a20ec-4025-4e8f-9def-7092d6e096cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:25.117871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a20ec-4025-4e8f-9def-7092d6e096cd service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:25.118160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a20ec-4025-4e8f-9def-7092d6e096cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:25.118438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a20ec-4025-4e8f-9def-7092d6e096cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:25.130732 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a20ec-4025-4e8f-9def-7092d6e096cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:25.132736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a20ec-4025-4e8f-9def-7092d6e096cd service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:25.144665 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1217/2415] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:25.682205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d419c95-fcca-4177-be82-d3277110ecde None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:25.688402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7d419c95-fcca-4177-be82-d3277110ecde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:25.701979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d419c95-fcca-4177-be82-d3277110ecde None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:25.707257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:25.708074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:25.708354 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:25.708624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] PATH_INFO: `/v3/projects/ff7fdf06df8942baae5674293f41788c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:25.710945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7fdf06df8942baae5674293f41788c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:25.712944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:25.817494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acb49-cb75-41e9-8ccf-627acc7835f7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:25.820891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acb49-cb75-41e9-8ccf-627acc7835f7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:25.820891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36acb49-cb75-41e9-8ccf-627acc7835f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:25.822227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:25.834332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7fdf06df8942baae5674293f41788c', 'request_id': 'req-5a9394a2-fde1-497b-bc92-43e3a7829206'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:25.840639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7fdf06df8942baae5674293f41788c', 'request_id': 'req-5a9394a2-fde1-497b-bc92-43e3a7829206'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:25.841584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e36acb49-cb75-41e9-8ccf-627acc7835f7 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:25.842791 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e36acb49-cb75-41e9-8ccf-627acc7835f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:25.847794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7fdf06df8942baae5674293f41788c', 'request_id': 'req-5a9394a2-fde1-497b-bc92-43e3a7829206'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:25.904021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1218/2416] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:42:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:42:25.910465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] The token cache is being invalidate because project ff7fdf06df8942baae5674293f41788c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:25.911612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:25.912656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:25.915928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1199/2417] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:25 2022] DELETE /identity/v3/projects/ff7fdf06df8942baae5674293f41788c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:25.917180 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36acb49-cb75-41e9-8ccf-627acc7835f7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:25.918591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e36acb49-cb75-41e9-8ccf-627acc7835f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:25.919990 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:25.921314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:25.985391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9394a2-fde1-497b-bc92-43e3a7829206 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:25.987618 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36acb49-cb75-41e9-8ccf-627acc7835f7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:25.991047 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:25.991952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:25.992932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:25.992932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] PATH_INFO: `/v3/users/4e7e7b9e51a74b1989bce1f983044602` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:25.993158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:25.993278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:25.993811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:25.994003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:25.994143 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e7e7b9e51a74b1989bce1f983044602)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:25.995823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:25.995945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:26.007296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:26.008865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:26.017560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1219/2418] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:42:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 03:42:26.271603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e7e7b9e51a74b1989bce1f983044602'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:26.299043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e7e7b9e51a74b1989bce1f983044602'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:26.299813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e7e7b9e51a74b1989bce1f983044602'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:26.300461 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e7e7b9e51a74b1989bce1f983044602'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:26.394600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e7e7b9e51a74b1989bce1f983044602'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:26.417823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1200/2419] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:42:25 2022] DELETE /identity/v3/users/4e7e7b9e51a74b1989bce1f983044602 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:27.091395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:27.104643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a326e-6c5d-4f81-9b7c-9211490d216d service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:27.171723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:27.228765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:27.230028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:27.230740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:27.231328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] PATH_INFO: `/v3/projects/13683de02d4c440fbdc1acd44fece9a0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:27.235428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13683de02d4c440fbdc1acd44fece9a0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:27.248150 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:27.402367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13683de02d4c440fbdc1acd44fece9a0', 'request_id': 'req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.410234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13683de02d4c440fbdc1acd44fece9a0', 'request_id': 'req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.417362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13683de02d4c440fbdc1acd44fece9a0', 'request_id': 'req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.520258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] The token cache is being invalidate because project 13683de02d4c440fbdc1acd44fece9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:27.520647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.521331 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.523717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1220/2420] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:27 2022] DELETE /identity/v3/projects/13683de02d4c440fbdc1acd44fece9a0 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:27.528450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:27.530787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:27.546268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43820a0-a636-4110-b488-bb3b7644cbe4 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:27.590774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:27.613951 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:27.615326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:27.615774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:27.616300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] PATH_INFO: `/v3/users/d88ff7656d4e4c719fe7e1b786f24b52` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:27.617926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88ff7656d4e4c719fe7e1b786f24b52)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:27.620347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:27.867443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88ff7656d4e4c719fe7e1b786f24b52'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.884640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88ff7656d4e4c719fe7e1b786f24b52'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.885490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88ff7656d4e4c719fe7e1b786f24b52'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.886211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88ff7656d4e4c719fe7e1b786f24b52'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.942870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-62e1248e-6355-4f70-832a-22cacedbb747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88ff7656d4e4c719fe7e1b786f24b52'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:27.959383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1201/2421] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:42:27 2022] DELETE /identity/v3/users/d88ff7656d4e4c719fe7e1b786f24b52 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:28.417846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:28.419976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:28.435673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaadb5-6ee9-4f61-91d1-cac21af2974a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:28.530985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:28.531890 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:28.532201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:28.532482 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] PATH_INFO: `/v3/projects/e66e3943aebf45cea7afb5f36a04d6a0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:28.533773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e66e3943aebf45cea7afb5f36a04d6a0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:28.535787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:28.673058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e66e3943aebf45cea7afb5f36a04d6a0', 'request_id': 'req-604f48e2-cefe-4674-8274-f218879297a8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:28.680988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e66e3943aebf45cea7afb5f36a04d6a0', 'request_id': 'req-604f48e2-cefe-4674-8274-f218879297a8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:28.690117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e66e3943aebf45cea7afb5f36a04d6a0', 'request_id': 'req-604f48e2-cefe-4674-8274-f218879297a8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:28.693058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:28.773832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] The token cache is being invalidate because project e66e3943aebf45cea7afb5f36a04d6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:28.774527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:28.775146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-604f48e2-cefe-4674-8274-f218879297a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:28.780987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1221/2422] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:28 2022] DELETE /identity/v3/projects/e66e3943aebf45cea7afb5f36a04d6a0 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:28.782578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054946-9fcd-4200-95e9-df90fbbd6068 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:28.783061 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054946-9fcd-4200-95e9-df90fbbd6068 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:28.783284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054946-9fcd-4200-95e9-df90fbbd6068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:28.824003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-81054946-9fcd-4200-95e9-df90fbbd6068 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:28.825975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-81054946-9fcd-4200-95e9-df90fbbd6068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:28.989197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1202/2423] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:28.991951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34170b-6baa-4640-8209-c9aeddf77c78 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:28.992848 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34170b-6baa-4640-8209-c9aeddf77c78 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:28.994697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34170b-6baa-4640-8209-c9aeddf77c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.026827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7d34170b-6baa-4640-8209-c9aeddf77c78 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:29.028698 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7d34170b-6baa-4640-8209-c9aeddf77c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:29.073370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054946-9fcd-4200-95e9-df90fbbd6068 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:29.076064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1222/2424] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:29.095806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054946-9fcd-4200-95e9-df90fbbd6068 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:29.110044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:29.111097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:29.111755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:29.112888 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.113766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:42:29.114094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:29.120015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:29.149974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1203/2425] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:42:29.151747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34170b-6baa-4640-8209-c9aeddf77c78 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:29.166094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34170b-6baa-4640-8209-c9aeddf77c78 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:29.171722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016420d3-b95a-4103-aada-3ce2682d3a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:29.172731 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016420d3-b95a-4103-aada-3ce2682d3a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:29.173286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016420d3-b95a-4103-aada-3ce2682d3a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:29.173749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016420d3-b95a-4103-aada-3ce2682d3a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.174745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016420d3-b95a-4103-aada-3ce2682d3a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:29.177417 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016420d3-b95a-4103-aada-3ce2682d3a85 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:29.276824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1223/2426] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:29 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:29.278130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:29.289334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cc4b6-2e3f-498c-b2ef-dbae08a062bb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:29.347032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:29.360828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8637518-372c-42c5-84b3-76ac54283202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:29.361572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8637518-372c-42c5-84b3-76ac54283202 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:29.361863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8637518-372c-42c5-84b3-76ac54283202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:29.362140 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8637518-372c-42c5-84b3-76ac54283202 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.362835 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8637518-372c-42c5-84b3-76ac54283202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:29.364780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8637518-372c-42c5-84b3-76ac54283202 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:29.438094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1204/2427] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:29 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:29.442179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016420d3-b95a-4103-aada-3ce2682d3a85 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:29.453040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016420d3-b95a-4103-aada-3ce2682d3a85 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:29.457687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0d1eb-4287-4ad7-a301-016f4dbb7311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:29.458335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0d1eb-4287-4ad7-a301-016f4dbb7311 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:29.458785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0d1eb-4287-4ad7-a301-016f4dbb7311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:29.459068 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0d1eb-4287-4ad7-a301-016f4dbb7311 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.460491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0d1eb-4287-4ad7-a301-016f4dbb7311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:29.461969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0d1eb-4287-4ad7-a301-016f4dbb7311 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:29.472309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1224/2428] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:29.476680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8637518-372c-42c5-84b3-76ac54283202 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:29.491918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8637518-372c-42c5-84b3-76ac54283202 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:29.503111 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9144b72-23e1-40aa-9f6d-78b773c17887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:29.503934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9144b72-23e1-40aa-9f6d-78b773c17887 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:29.504287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9144b72-23e1-40aa-9f6d-78b773c17887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:29.504825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9144b72-23e1-40aa-9f6d-78b773c17887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.505901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9144b72-23e1-40aa-9f6d-78b773c17887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:29.508314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9144b72-23e1-40aa-9f6d-78b773c17887 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:29.525234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1205/2429] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:29.526489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0d1eb-4287-4ad7-a301-016f4dbb7311 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:29.539114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0d1eb-4287-4ad7-a301-016f4dbb7311 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:29.553984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:29.554917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:29.555280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:29.555493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] PATH_INFO: `/v3/projects/ff4caa4939364a39b4f65bf35f533491/users/fb6663516e3a42fb911ef967394ce7f1/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.572717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4caa4939364a39b4f65bf35f533491, user_id=fb6663516e3a42fb911ef967394ce7f1, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:29.575976 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:29.667571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1225/2430] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:29 2022] PUT /identity/v3/projects/ff4caa4939364a39b4f65bf35f533491/users/fb6663516e3a42fb911ef967394ce7f1/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:29.680403 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20289e-fa54-426c-a34d-d218ad33f783 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:29.680703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20289e-fa54-426c-a34d-d218ad33f783 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:29.681156 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20289e-fa54-426c-a34d-d218ad33f783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.736556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2f20289e-fa54-426c-a34d-d218ad33f783 None None] MFA Rules not processed for user `fb6663516e3a42fb911ef967394ce7f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:29.738082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2f20289e-fa54-426c-a34d-d218ad33f783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:29.812115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1206/2431] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:29.824190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:29.825972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:29.889756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db8f5-c487-41fe-a2ad-f0c075c1b184 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:29.893521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e3f43-b5de-4492-9689-75f7d93e37ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:29.894137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3f43-b5de-4492-9689-75f7d93e37ea service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:29.894426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3f43-b5de-4492-9689-75f7d93e37ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:29.894712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3f43-b5de-4492-9689-75f7d93e37ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:29.908157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e3f43-b5de-4492-9689-75f7d93e37ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:29.909680 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e3f43-b5de-4492-9689-75f7d93e37ea service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:29.927383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1226/2432] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:29.948308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20289e-fa54-426c-a34d-d218ad33f783 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:30.015268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20289e-fa54-426c-a34d-d218ad33f783 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:30.020518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:30.021428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:30.021708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:30.021980 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] PATH_INFO: `/v3/users/fb6663516e3a42fb911ef967394ce7f1` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:30.023217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb6663516e3a42fb911ef967394ce7f1)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:30.025178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:30.265649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb6663516e3a42fb911ef967394ce7f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:30.285488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:30.294601 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb6663516e3a42fb911ef967394ce7f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:30.295332 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb6663516e3a42fb911ef967394ce7f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:30.295939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb6663516e3a42fb911ef967394ce7f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:30.412200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb6663516e3a42fb911ef967394ce7f1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:30.451268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1207/2433] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:42:29 2022] DELETE /identity/v3/users/fb6663516e3a42fb911ef967394ce7f1 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:30.453488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb8545-0ff0-4507-b5f5-40180a5e0825 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:30.454099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb8545-0ff0-4507-b5f5-40180a5e0825 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:30.454099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb8545-0ff0-4507-b5f5-40180a5e0825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:30.472964 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b9bb8545-0ff0-4507-b5f5-40180a5e0825 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:30.474214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b9bb8545-0ff0-4507-b5f5-40180a5e0825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:30.578699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1227/2434] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:30.590211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:30.591787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:30.663804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15e495-102e-4fae-b59a-2aa956a908a5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:30.676617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcaa381-0772-4420-9a40-c302c949c872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:30.686822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaa381-0772-4420-9a40-c302c949c872 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:30.686822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaa381-0772-4420-9a40-c302c949c872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:30.686822 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaa381-0772-4420-9a40-c302c949c872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:30.693287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcaa381-0772-4420-9a40-c302c949c872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:30.694915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcaa381-0772-4420-9a40-c302c949c872 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:30.706240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1208/2435] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:31.973846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb8545-0ff0-4507-b5f5-40180a5e0825 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:31.980518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b9bb8545-0ff0-4507-b5f5-40180a5e0825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:31.993895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb8545-0ff0-4507-b5f5-40180a5e0825 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:32.004279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:32.005038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:32.005370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:32.005706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] PATH_INFO: `/v3/projects/ff4caa4939364a39b4f65bf35f533491` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:32.006965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4caa4939364a39b4f65bf35f533491)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:32.008948 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:32.172515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:32.191356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4caa4939364a39b4f65bf35f533491', 'request_id': 'req-116384b0-621c-4b8f-916b-50a22e076324'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:32.199073 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4caa4939364a39b4f65bf35f533491', 'request_id': 'req-116384b0-621c-4b8f-916b-50a22e076324'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:32.205881 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4caa4939364a39b4f65bf35f533491', 'request_id': 'req-116384b0-621c-4b8f-916b-50a22e076324'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:32.314393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] The token cache is being invalidate because project ff4caa4939364a39b4f65bf35f533491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:32.314879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:32.315642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-116384b0-621c-4b8f-916b-50a22e076324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:32.321905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1228/2436] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:31 2022] DELETE /identity/v3/projects/ff4caa4939364a39b4f65bf35f533491 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:32.324603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41791a-ab08-4488-bcc3-0e091f195b1f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:32.325261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41791a-ab08-4488-bcc3-0e091f195b1f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:32.325468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41791a-ab08-4488-bcc3-0e091f195b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:32.347730 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4e41791a-ab08-4488-bcc3-0e091f195b1f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:32.349276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4e41791a-ab08-4488-bcc3-0e091f195b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:32.423111 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1209/2437] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:32.425253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa444-1ff1-4d07-9573-8f060fefa4ca None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:32.425557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa444-1ff1-4d07-9573-8f060fefa4ca None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:32.425843 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa444-1ff1-4d07-9573-8f060fefa4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:32.447531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-238aa444-1ff1-4d07-9573-8f060fefa4ca None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:32.448774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-238aa444-1ff1-4d07-9573-8f060fefa4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:32.458488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1229/2438] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:32.463254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41791a-ab08-4488-bcc3-0e091f195b1f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:32.479216 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41791a-ab08-4488-bcc3-0e091f195b1f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:32.486460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:32.489330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:32.489330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:32.489330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:32.489330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:42:32.489330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:32.499254 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:32.515068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1210/2439] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:42:32.519231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa444-1ff1-4d07-9573-8f060fefa4ca None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:32.531929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa444-1ff1-4d07-9573-8f060fefa4ca None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:32.543004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:32.543673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:32.543952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:32.544230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:32.544917 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:32.546868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:32.649457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1230/2440] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:32 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:42:32.652572 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:32.663367 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a306f27-210d-4fdd-99d6-6b01a115b38d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:32.787153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daca6ca-dc70-41ca-824f-00cb3f1db330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:32.788069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daca6ca-dc70-41ca-824f-00cb3f1db330 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:32.788329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:32.788447 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daca6ca-dc70-41ca-824f-00cb3f1db330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:32.788625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daca6ca-dc70-41ca-824f-00cb3f1db330 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:32.789666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daca6ca-dc70-41ca-824f-00cb3f1db330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:32.790143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:32.791680 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daca6ca-dc70-41ca-824f-00cb3f1db330 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:32.806289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:32.872025 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df4a42-5e80-48a1-b1fe-dadb79c6675f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:32.885417 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:32.886804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:32.888909 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:32.889454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:32.892330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:32.906786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1211/2441] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:32 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:32.906786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daca6ca-dc70-41ca-824f-00cb3f1db330 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:32.916800 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:32.919186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:32.924622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daca6ca-dc70-41ca-824f-00cb3f1db330 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:32.933000 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1231/2442] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:32.946817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fe10c-fdb2-4bc2-969c-428309b25600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:32.947848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fe10c-fdb2-4bc2-969c-428309b25600 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:32.948218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fe10c-fdb2-4bc2-969c-428309b25600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:32.948573 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fe10c-fdb2-4bc2-969c-428309b25600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:32.949432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fe10c-fdb2-4bc2-969c-428309b25600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:32.951385 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fe10c-fdb2-4bc2-969c-428309b25600 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:32.963660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1212/2443] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:32.971647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:32.993577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9f9c-d3f9-4265-822e-8db64d8fb9e8 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:33.002348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac43f7-3a48-410b-a16d-999f491e0b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:33.003710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43f7-3a48-410b-a16d-999f491e0b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:33.004328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43f7-3a48-410b-a16d-999f491e0b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:33.004815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43f7-3a48-410b-a16d-999f491e0b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:33.008457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac43f7-3a48-410b-a16d-999f491e0b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:33.010868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac43f7-3a48-410b-a16d-999f491e0b44 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:33.023672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1232/2444] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:33.030684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fe10c-fdb2-4bc2-969c-428309b25600 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:33.046574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fe10c-fdb2-4bc2-969c-428309b25600 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:33.064330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:33.066132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:33.068270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:33.068270 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] PATH_INFO: `/v3/projects/23f8194639b14942b36011698807b031/users/c1a9600bad844f60937aa35126cd1be4/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:33.099937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f8194639b14942b36011698807b031, user_id=c1a9600bad844f60937aa35126cd1be4, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:33.103817 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:33.185345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1213/2445] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:33 2022] PUT /identity/v3/projects/23f8194639b14942b36011698807b031/users/c1a9600bad844f60937aa35126cd1be4/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:33.187368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f377dc0-8bd3-44ac-802e-a93f1533c2ea None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:33.188020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f377dc0-8bd3-44ac-802e-a93f1533c2ea None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:33.188712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f377dc0-8bd3-44ac-802e-a93f1533c2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:33.247804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1f377dc0-8bd3-44ac-802e-a93f1533c2ea None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:33.249275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1f377dc0-8bd3-44ac-802e-a93f1533c2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:33.414006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1233/2446] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:33.425141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:33.434632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:33.515494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63879f44-143a-4cf7-a08c-e674094ad497 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:33.519843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7b092-273e-478a-841f-38801e8939a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:33.520499 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b092-273e-478a-841f-38801e8939a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:33.520784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b092-273e-478a-841f-38801e8939a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:33.521076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b092-273e-478a-841f-38801e8939a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:33.546032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7b092-273e-478a-841f-38801e8939a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:33.549020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7b092-273e-478a-841f-38801e8939a2 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:33.578036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1214/2447] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:38.320862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c21009-90fc-4a1d-b40f-d6386a463f9a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:38.321419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c21009-90fc-4a1d-b40f-d6386a463f9a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:38.321578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c21009-90fc-4a1d-b40f-d6386a463f9a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:38.323280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1234/2448] 10.222.0.41 () {54 vars in 921 bytes} [Sat May 28 03:42:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 03:42:38.338713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ad75c-735d-446d-8d7d-bd66b7dc72e9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:38.339290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ad75c-735d-446d-8d7d-bd66b7dc72e9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:38.339290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ad75c-735d-446d-8d7d-bd66b7dc72e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:38.421696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-498ad75c-735d-446d-8d7d-bd66b7dc72e9 None None] MFA Rules not processed for user `1e771dc83fa34d79947647e56ca338a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:38.425936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-498ad75c-735d-446d-8d7d-bd66b7dc72e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:38.495237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c21009-90fc-4a1d-b40f-d6386a463f9a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:38.497728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-68c21009-90fc-4a1d-b40f-d6386a463f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:38.525371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c21009-90fc-4a1d-b40f-d6386a463f9a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:38.533941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:38.538651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:38.542714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:38.542714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] PATH_INFO: `/v3/users/dc6a3335562448d1934fa18e8f51b119` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:38.542714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6a3335562448d1934fa18e8f51b119)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:38.544871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:38.545874 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1215/2449] 10.222.0.41 () {58 vars in 1018 bytes} [Sat May 28 03:42:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:38.563454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:38.567720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ad75c-735d-446d-8d7d-bd66b7dc72e9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:38.570662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-498ad75c-735d-446d-8d7d-bd66b7dc72e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:38.596962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ad75c-735d-446d-8d7d-bd66b7dc72e9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:38.608055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74a160-0a48-45dc-aea0-17798e875d63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:38.608719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a160-0a48-45dc-aea0-17798e875d63 service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:38.609017 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a160-0a48-45dc-aea0-17798e875d63 service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:38.609310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74a160-0a48-45dc-aea0-17798e875d63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:38.619352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74a160-0a48-45dc-aea0-17798e875d63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:38.621128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74a160-0a48-45dc-aea0-17798e875d63 service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:38.635807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1216/2450] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:38.648548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74a160-0a48-45dc-aea0-17798e875d63 service placement] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:38.662366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74a160-0a48-45dc-aea0-17798e875d63 service placement] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:38.665698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:38.666418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:38.666805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:38.667147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:38.670440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:38.715189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:38.765464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6a3335562448d1934fa18e8f51b119'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:38.780605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6a3335562448d1934fa18e8f51b119'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:38.786628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:38.788149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:38.811005 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6a3335562448d1934fa18e8f51b119'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:38.811434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1217/2451] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:42:38.811785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6a3335562448d1934fa18e8f51b119'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:38.879586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe586407-d47e-4c53-a794-883e1d5592f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6a3335562448d1934fa18e8f51b119'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:38.893642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1235/2452] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:42:38 2022] DELETE /identity/v3/users/dc6a3335562448d1934fa18e8f51b119 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:39.397816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:39.410337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2b055-cdc7-44cf-b4ed-47053f783c93 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:39.499897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:39.502066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:39.502518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:39.502945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] PATH_INFO: `/v3/projects/70eaf602c8db4ec1bfb26e5eb3b13c82` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:39.507135 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] RBAC: Authorizing `identity:delete_project(project_id=70eaf602c8db4ec1bfb26e5eb3b13c82)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:39.509978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:39.661493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'request_id': 'req-64c93be6-2171-44b3-aee2-ded42b5f389c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:39.677897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'request_id': 'req-64c93be6-2171-44b3-aee2-ded42b5f389c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:39.684884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70eaf602c8db4ec1bfb26e5eb3b13c82', 'request_id': 'req-64c93be6-2171-44b3-aee2-ded42b5f389c'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:39.785192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] The token cache is being invalidate because project 70eaf602c8db4ec1bfb26e5eb3b13c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:42:39.788787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:39.790423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-64c93be6-2171-44b3-aee2-ded42b5f389c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:42:39.799280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc7ca-a394-49e9-a83d-4d4efe66b4b2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:39.799885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1218/2453] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:42:39 2022] DELETE /identity/v3/projects/70eaf602c8db4ec1bfb26e5eb3b13c82 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:39.800006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc7ca-a394-49e9-a83d-4d4efe66b4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:39.800193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cc7ca-a394-49e9-a83d-4d4efe66b4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:39.827423 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-339cc7ca-a394-49e9-a83d-4d4efe66b4b2 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:39.829537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-339cc7ca-a394-49e9-a83d-4d4efe66b4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:39.924451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1236/2454] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:39.935139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602676e7-2aa6-4807-b6a8-014a1fe1a7a1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:39.935553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602676e7-2aa6-4807-b6a8-014a1fe1a7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:39.935845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602676e7-2aa6-4807-b6a8-014a1fe1a7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:39.959408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-602676e7-2aa6-4807-b6a8-014a1fe1a7a1 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:39.960766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-602676e7-2aa6-4807-b6a8-014a1fe1a7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:39.978485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cc7ca-a394-49e9-a83d-4d4efe66b4b2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:39.978675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1219/2455] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:39.990842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cc7ca-a394-49e9-a83d-4d4efe66b4b2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:40.004711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:40.004711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:40.005010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:40.005931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:40.009893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:42:40.012348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:40.012348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:40.030836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602676e7-2aa6-4807-b6a8-014a1fe1a7a1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:40.041159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1237/2456] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:42:40.044379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602676e7-2aa6-4807-b6a8-014a1fe1a7a1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:40.050203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983ba68-4d7b-4eb1-b808-c58749f96a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:40.059229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983ba68-4d7b-4eb1-b808-c58749f96a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:40.059849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983ba68-4d7b-4eb1-b808-c58749f96a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:40.060358 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983ba68-4d7b-4eb1-b808-c58749f96a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:40.061556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983ba68-4d7b-4eb1-b808-c58749f96a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:40.063978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983ba68-4d7b-4eb1-b808-c58749f96a27 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:40.173645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:40.174759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1220/2457] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:40 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:40.187110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835219e-a313-43b7-a4c5-fc6bc9e33404 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:40.279642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1ed0b-ba95-4bce-a76b-d49a58d6d4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:40.280460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1ed0b-ba95-4bce-a76b-d49a58d6d4af admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:40.280832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1ed0b-ba95-4bce-a76b-d49a58d6d4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:40.281207 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1ed0b-ba95-4bce-a76b-d49a58d6d4af admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:40.281974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1ed0b-ba95-4bce-a76b-d49a58d6d4af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:40.284021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1ed0b-ba95-4bce-a76b-d49a58d6d4af admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:40.412277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1238/2458] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:40 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:42:40.416869 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983ba68-4d7b-4eb1-b808-c58749f96a27 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:40.428734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983ba68-4d7b-4eb1-b808-c58749f96a27 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:40.434344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecadd3-a5fa-496b-81dd-fad382610e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:40.435142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecadd3-a5fa-496b-81dd-fad382610e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:40.438452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecadd3-a5fa-496b-81dd-fad382610e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:40.438452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecadd3-a5fa-496b-81dd-fad382610e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:40.438452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecadd3-a5fa-496b-81dd-fad382610e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:40.438452 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecadd3-a5fa-496b-81dd-fad382610e0c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:40.460900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1ed0b-ba95-4bce-a76b-d49a58d6d4af admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:40.469693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1221/2459] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:40.479166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1ed0b-ba95-4bce-a76b-d49a58d6d4af admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:40.488056 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948182e-7eac-466d-a637-5d8182488988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:40.489028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948182e-7eac-466d-a637-5d8182488988 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:40.490013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948182e-7eac-466d-a637-5d8182488988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:40.490574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948182e-7eac-466d-a637-5d8182488988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:40.491555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948182e-7eac-466d-a637-5d8182488988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:40.494752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948182e-7eac-466d-a637-5d8182488988 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:40.500568 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:40.515875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecadd3-a5fa-496b-81dd-fad382610e0c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:40.536853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1239/2460] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:40.539978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecadd3-a5fa-496b-81dd-fad382610e0c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:40.548815 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0c931-717d-45ee-a5d8-a9bcbdf4de88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:40.554491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0c931-717d-45ee-a5d8-a9bcbdf4de88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:40.555085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0c931-717d-45ee-a5d8-a9bcbdf4de88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:40.555696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0c931-717d-45ee-a5d8-a9bcbdf4de88 admin admin] PATH_INFO: `/v3/projects/c7f2e71c1ea8422b9f13d34056f29a9d/users/760513191e2d4b38b3dfb1bb7c1a89f5/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:40.575813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:40.580190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0c931-717d-45ee-a5d8-a9bcbdf4de88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7f2e71c1ea8422b9f13d34056f29a9d, user_id=760513191e2d4b38b3dfb1bb7c1a89f5, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:40.583185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0c931-717d-45ee-a5d8-a9bcbdf4de88 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:40.707308 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1222/2461] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:40 2022] PUT /identity/v3/projects/c7f2e71c1ea8422b9f13d34056f29a9d/users/760513191e2d4b38b3dfb1bb7c1a89f5/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:40.717085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824279-39b4-4958-b714-c335fb372776 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:40.717438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824279-39b4-4958-b714-c335fb372776 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:40.717691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2824279-39b4-4958-b714-c335fb372776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:40.750003 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d2824279-39b4-4958-b714-c335fb372776 None None] MFA Rules not processed for user `760513191e2d4b38b3dfb1bb7c1a89f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:40.751530 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d2824279-39b4-4958-b714-c335fb372776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:40.896721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1240/2462] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:40.897949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0c931-717d-45ee-a5d8-a9bcbdf4de88 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:40.972507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a3db2-27db-4865-b804-a98086d55b28 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:40.973385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a3db2-27db-4865-b804-a98086d55b28 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:40.973826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a3db2-27db-4865-b804-a98086d55b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.016590 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-128a3db2-27db-4865-b804-a98086d55b28 None None] MFA Rules not processed for user `c1a9600bad844f60937aa35126cd1be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:41.019376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-128a3db2-27db-4865-b804-a98086d55b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:41.035608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0c931-717d-45ee-a5d8-a9bcbdf4de88 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:41.040598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04328cf1-45c6-4515-aadd-828a131dbcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:41.042399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04328cf1-45c6-4515-aadd-828a131dbcbb admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:41.042829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04328cf1-45c6-4515-aadd-828a131dbcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:41.043115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04328cf1-45c6-4515-aadd-828a131dbcbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.043952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04328cf1-45c6-4515-aadd-828a131dbcbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:41.046376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04328cf1-45c6-4515-aadd-828a131dbcbb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:41.125414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1241/2463] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:41.130739 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a3db2-27db-4865-b804-a98086d55b28 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:41.134596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-128a3db2-27db-4865-b804-a98086d55b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:41.159471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1223/2464] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:40 2022] POST /identity/v3/projects => generated 373 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:41.160847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04328cf1-45c6-4515-aadd-828a131dbcbb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:41.165619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a3db2-27db-4865-b804-a98086d55b28 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:41.313480 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763884d7-5fec-445b-a35e-98d0b7db5609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:41.314250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763884d7-5fec-445b-a35e-98d0b7db5609 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:41.314612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763884d7-5fec-445b-a35e-98d0b7db5609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:41.315030 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763884d7-5fec-445b-a35e-98d0b7db5609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.315802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763884d7-5fec-445b-a35e-98d0b7db5609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:41.316895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04328cf1-45c6-4515-aadd-828a131dbcbb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:41.318082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763884d7-5fec-445b-a35e-98d0b7db5609 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:41.323710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96045588-2b2e-42d8-a275-359503990174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:41.324569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96045588-2b2e-42d8-a275-359503990174 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:41.325009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96045588-2b2e-42d8-a275-359503990174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:41.325394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96045588-2b2e-42d8-a275-359503990174 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.326258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96045588-2b2e-42d8-a275-359503990174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:41.328338 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96045588-2b2e-42d8-a275-359503990174 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:41.457529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1242/2465] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:42:41 2022] POST /identity/v3/projects => generated 385 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:41.462190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763884d7-5fec-445b-a35e-98d0b7db5609 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:41.468190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:41.481788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1224/2466] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:41 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:42:41.486438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96045588-2b2e-42d8-a275-359503990174 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:41.505214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96045588-2b2e-42d8-a275-359503990174 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:41.592742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763884d7-5fec-445b-a35e-98d0b7db5609 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:41.599203 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8bd57-f265-4735-8ce9-e59defec8b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:41.601383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8bd57-f265-4735-8ce9-e59defec8b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:41.601671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8bd57-f265-4735-8ce9-e59defec8b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:41.602416 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf8bd57-f265-4735-8ce9-e59defec8b38 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.602655 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8bd57-f265-4735-8ce9-e59defec8b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:41.606102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf8bd57-f265-4735-8ce9-e59defec8b38 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:41.623314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266f80-30a2-4a2e-81a7-589e8ff615a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:41.624498 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266f80-30a2-4a2e-81a7-589e8ff615a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:41.625257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266f80-30a2-4a2e-81a7-589e8ff615a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:41.625747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266f80-30a2-4a2e-81a7-589e8ff615a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.626968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72266f80-30a2-4a2e-81a7-589e8ff615a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:41.629086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72266f80-30a2-4a2e-81a7-589e8ff615a7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:41.643223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1225/2467] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:41.647805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266f80-30a2-4a2e-81a7-589e8ff615a7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:41.653444 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:41.659038 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266f80-30a2-4a2e-81a7-589e8ff615a7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:41.665147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f454e-f2c7-4f58-9d98-5295b08cd129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:41.665355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f454e-f2c7-4f58-9d98-5295b08cd129 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:41.665666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f454e-f2c7-4f58-9d98-5295b08cd129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:41.665896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f454e-f2c7-4f58-9d98-5295b08cd129 admin admin] PATH_INFO: `/v3/projects/dc3eae34c82744f6a4221a942d434d6f/users/2c2440089c534450b24845482a7fb0bb/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.682716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f454e-f2c7-4f58-9d98-5295b08cd129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3eae34c82744f6a4221a942d434d6f, user_id=2c2440089c534450b24845482a7fb0bb, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:41.685430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f454e-f2c7-4f58-9d98-5295b08cd129 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:41.729696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1243/2468] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:42:41 2022] POST /identity/v3/users => generated 299 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:42:41.731450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8bd57-f265-4735-8ce9-e59defec8b38 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:41.747068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1226/2469] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:41 2022] PUT /identity/v3/projects/dc3eae34c82744f6a4221a942d434d6f/users/2c2440089c534450b24845482a7fb0bb/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:42:41.753578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f454e-f2c7-4f58-9d98-5295b08cd129 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:41.755291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf8bd57-f265-4735-8ce9-e59defec8b38 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:41.920558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f454e-f2c7-4f58-9d98-5295b08cd129 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:41.927109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427aebfa-8ce0-4d63-abd9-e1965fc6e587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:41.928164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aebfa-8ce0-4d63-abd9-e1965fc6e587 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:41.928164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aebfa-8ce0-4d63-abd9-e1965fc6e587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:41.928364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427aebfa-8ce0-4d63-abd9-e1965fc6e587 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.929116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427aebfa-8ce0-4d63-abd9-e1965fc6e587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:41.930805 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427aebfa-8ce0-4d63-abd9-e1965fc6e587 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:41.945479 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa8cc-8cf8-4173-9bad-056e0b64a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:41.946185 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa8cc-8cf8-4173-9bad-056e0b64a6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:41.946466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa8cc-8cf8-4173-9bad-056e0b64a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:41.946742 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beaa8cc-8cf8-4173-9bad-056e0b64a6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:41.947486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaa8cc-8cf8-4173-9bad-056e0b64a6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:41.952842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beaa8cc-8cf8-4173-9bad-056e0b64a6d6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:41.963927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1244/2470] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:41.969381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427aebfa-8ce0-4d63-abd9-e1965fc6e587 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:41.981841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1227/2471] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:42:41.989901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa8cc-8cf8-4173-9bad-056e0b64a6d6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:42.002354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427aebfa-8ce0-4d63-abd9-e1965fc6e587 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:42.012175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beaa8cc-8cf8-4173-9bad-056e0b64a6d6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:42.015761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aede00-b982-4ca0-b059-b93c2bf8cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:42.016839 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aede00-b982-4ca0-b059-b93c2bf8cff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:42.018701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aede00-b982-4ca0-b059-b93c2bf8cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:42.019663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aede00-b982-4ca0-b059-b93c2bf8cff8 admin admin] PATH_INFO: `/v3/projects/4e6febafbcf34c60bb33ade27a42b83e/users/a61e06424d9d459286e0c33c26a534e5/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:42.029899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3b063-995a-4fcf-9314-e13f9fc3032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:42.031052 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b063-995a-4fcf-9314-e13f9fc3032b admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:42.031435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b063-995a-4fcf-9314-e13f9fc3032b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:42.031818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3b063-995a-4fcf-9314-e13f9fc3032b admin admin] PATH_INFO: `/v3/projects/dc3eae34c82744f6a4221a942d434d6f/users/2c2440089c534450b24845482a7fb0bb/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:42.040236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3b063-995a-4fcf-9314-e13f9fc3032b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3eae34c82744f6a4221a942d434d6f, user_id=2c2440089c534450b24845482a7fb0bb, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:42.043267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3b063-995a-4fcf-9314-e13f9fc3032b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:42.055891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aede00-b982-4ca0-b059-b93c2bf8cff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6febafbcf34c60bb33ade27a42b83e, user_id=a61e06424d9d459286e0c33c26a534e5, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:42.059220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aede00-b982-4ca0-b059-b93c2bf8cff8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:42.072899 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1228/2472] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:41 2022] PUT /identity/v3/projects/dc3eae34c82744f6a4221a942d434d6f/users/2c2440089c534450b24845482a7fb0bb/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:42.086754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2401a60-7142-4d82-9269-ce1fb8f69b9d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:42.087679 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2401a60-7142-4d82-9269-ce1fb8f69b9d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:42.088083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2401a60-7142-4d82-9269-ce1fb8f69b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:42.161810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f2401a60-7142-4d82-9269-ce1fb8f69b9d None None] MFA Rules not processed for user `2c2440089c534450b24845482a7fb0bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:42.161810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f2401a60-7142-4d82-9269-ce1fb8f69b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:42.280239 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1229/2473] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:42:42.297610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2401a60-7142-4d82-9269-ce1fb8f69b9d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:42.310072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f2401a60-7142-4d82-9269-ce1fb8f69b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:42.382777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:42.396091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2401a60-7142-4d82-9269-ce1fb8f69b9d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:42.405223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2463343-3de1-4970-a022-cf46d9b6eba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:42.405658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2463343-3de1-4970-a022-cf46d9b6eba1 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:42.406212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2463343-3de1-4970-a022-cf46d9b6eba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:42.406598 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2463343-3de1-4970-a022-cf46d9b6eba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:42.430984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2463343-3de1-4970-a022-cf46d9b6eba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:42.437136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2463343-3de1-4970-a022-cf46d9b6eba1 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:42.462022 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1230/2474] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:42.641049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1245/2475] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:41 2022] PUT /identity/v3/projects/4e6febafbcf34c60bb33ade27a42b83e/users/a61e06424d9d459286e0c33c26a534e5/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:42.650424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2463343-3de1-4970-a022-cf46d9b6eba1 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:42.758168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2463343-3de1-4970-a022-cf46d9b6eba1 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:42.848776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:42.849418 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:42.849782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:42.850158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:42.851570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:42.853266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:42.870960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1231/2476] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:42:42.874529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aede00-b982-4ca0-b059-b93c2bf8cff8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:42.885555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aede00-b982-4ca0-b059-b93c2bf8cff8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:42.890563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc09aa-51d3-48f9-994b-6e7fcd4fb035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:42.891542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc09aa-51d3-48f9-994b-6e7fcd4fb035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:42.891937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc09aa-51d3-48f9-994b-6e7fcd4fb035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:42.892290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc09aa-51d3-48f9-994b-6e7fcd4fb035 admin admin] PATH_INFO: `/v3/projects/4e6febafbcf34c60bb33ade27a42b83e/users/a61e06424d9d459286e0c33c26a534e5/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:42.894386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc09aa-51d3-48f9-994b-6e7fcd4fb035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e6febafbcf34c60bb33ade27a42b83e, user_id=a61e06424d9d459286e0c33c26a534e5, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:42.903143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc09aa-51d3-48f9-994b-6e7fcd4fb035 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:42.917004 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1246/2477] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:42:42 2022] PUT /identity/v3/projects/4e6febafbcf34c60bb33ade27a42b83e/users/a61e06424d9d459286e0c33c26a534e5/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:42:42.953039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:42.955203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:43.034671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212be1f-3602-49c4-b4b6-e9ef305ff64e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:43.103307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:43.122793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:43.123812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:43.124853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:43.125605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:43.188315 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:43.189859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:43.198824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1232/2478] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:43.275044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0899f1-8449-4cc3-9ce9-2f96ab96dedc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:43.275784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0899f1-8449-4cc3-9ce9-2f96ab96dedc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:43.275784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0899f1-8449-4cc3-9ce9-2f96ab96dedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:43.298799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1a0899f1-8449-4cc3-9ce9-2f96ab96dedc None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:43.300152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1a0899f1-8449-4cc3-9ce9-2f96ab96dedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:43.355860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:43.366934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1247/2479] 10.222.0.41 () {58 vars in 967 bytes} [Sat May 28 03:42:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:43.378995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:43.380417 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:43.463728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554642d9-c061-4cec-a85d-61b36338a16a service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:43.469006 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:43.469695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:43.469975 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:43.470252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:43.481854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:43.482977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:43.492305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1233/2480] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:42:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 03:42:49.347729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35b14-b5f5-4f59-b01f-8f34310066df None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:49.348234 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35b14-b5f5-4f59-b01f-8f34310066df None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:49.348318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e35b14-b5f5-4f59-b01f-8f34310066df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:49.384437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c3e35b14-b5f5-4f59-b01f-8f34310066df None None] MFA Rules not processed for user `a61e06424d9d459286e0c33c26a534e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:49.385813 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c3e35b14-b5f5-4f59-b01f-8f34310066df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:49.484447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1248/2481] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:42:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:49.498174 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:49.506938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:49.565210 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdf819-f4ce-4b04-a02d-896b5b866894 service swift] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:49.569429 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0867a-249f-46a5-96b7-0061ca6b2ea3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a47589bf9e7c4acba200979e3a91a6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:49.570078 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0867a-249f-46a5-96b7-0061ca6b2ea3 service glance] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:49.570362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0867a-249f-46a5-96b7-0061ca6b2ea3 service glance] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:49.570735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0867a-249f-46a5-96b7-0061ca6b2ea3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:49.618262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:42:49.632279 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0867a-249f-46a5-96b7-0061ca6b2ea3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:49.633780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0867a-249f-46a5-96b7-0061ca6b2ea3 service glance] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:49.649377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1234/2482] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:49.899236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058303d-19b6-4b8b-9d21-f5bf22495a9b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:49.905176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058303d-19b6-4b8b-9d21-f5bf22495a9b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:49.905176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058303d-19b6-4b8b-9d21-f5bf22495a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:49.925689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0058303d-19b6-4b8b-9d21-f5bf22495a9b None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:42:49.926904 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0058303d-19b6-4b8b-9d21-f5bf22495a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:49.934983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1249/2483] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:42:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:42:49.947730 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0867a-249f-46a5-96b7-0061ca6b2ea3 service glance] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:49.961711 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0867a-249f-46a5-96b7-0061ca6b2ea3 service glance] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:49.966847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552d22a-655a-4277-b9dd-487c5ffdfbb1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:49.967497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552d22a-655a-4277-b9dd-487c5ffdfbb1 service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:49.967783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552d22a-655a-4277-b9dd-487c5ffdfbb1 service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:49.968068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552d22a-655a-4277-b9dd-487c5ffdfbb1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:49.977645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552d22a-655a-4277-b9dd-487c5ffdfbb1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:49.981525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552d22a-655a-4277-b9dd-487c5ffdfbb1 service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:49.989611 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1235/2484] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:42:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:42:59.458037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058303d-19b6-4b8b-9d21-f5bf22495a9b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:42:59.460691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0058303d-19b6-4b8b-9d21-f5bf22495a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:59.539386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058303d-19b6-4b8b-9d21-f5bf22495a9b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:42:59.544555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:42:59.545249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:42:59.545540 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:42:59.545825 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:42:59.548425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:42:59.632385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:42:59.633807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:42:59.644478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1250/2485] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:42:59.985823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=12 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:02.156914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd2924-a3c5-4f28-82eb-332a1fd68fc8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:02.157848 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd2924-a3c5-4f28-82eb-332a1fd68fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:02.158629 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd2924-a3c5-4f28-82eb-332a1fd68fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:02.182043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-cbdd2924-a3c5-4f28-82eb-332a1fd68fc8 None None] MFA Rules not processed for user `c1a9600bad844f60937aa35126cd1be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:02.183355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cbdd2924-a3c5-4f28-82eb-332a1fd68fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:02.190458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1236/2486] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:03.902292 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:03.910886 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:03.925148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b5672-ce2b-46ad-be5a-da3d55bbbeb0 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:03.930750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:03.931639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:03.931914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:03.932191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] PATH_INFO: `/v3/users/9d420e4b554e4deab21b89094e0547ca` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:03.933440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d420e4b554e4deab21b89094e0547ca)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:03.935388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:03.958541 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:04.128611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d420e4b554e4deab21b89094e0547ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.136290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d420e4b554e4deab21b89094e0547ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.151027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d420e4b554e4deab21b89094e0547ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.152151 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d420e4b554e4deab21b89094e0547ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.254117 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-22f1ff43-e45d-4858-9b68-2ae5313eaf03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d420e4b554e4deab21b89094e0547ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.259126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1251/2487] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:03 2022] DELETE /identity/v3/users/9d420e4b554e4deab21b89094e0547ca => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:04.504546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd2924-a3c5-4f28-82eb-332a1fd68fc8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:04.516842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd2924-a3c5-4f28-82eb-332a1fd68fc8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:04.575175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:04.576084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:04.576375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:04.576653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] PATH_INFO: `/v3/projects/68e1a9e256f64e46919b1e362b4f68d8` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:04.577855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e1a9e256f64e46919b1e362b4f68d8)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:04.579769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:04.697857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e1a9e256f64e46919b1e362b4f68d8', 'request_id': 'req-f3fb8768-b1d7-4378-bb52-2bd16491d923'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.703944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e1a9e256f64e46919b1e362b4f68d8', 'request_id': 'req-f3fb8768-b1d7-4378-bb52-2bd16491d923'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.710031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e1a9e256f64e46919b1e362b4f68d8', 'request_id': 'req-f3fb8768-b1d7-4378-bb52-2bd16491d923'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.793482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] The token cache is being invalidate because project 68e1a9e256f64e46919b1e362b4f68d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:04.793772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.794392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f3fb8768-b1d7-4378-bb52-2bd16491d923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:04.795918 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1237/2488] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:04 2022] DELETE /identity/v3/projects/68e1a9e256f64e46919b1e362b4f68d8 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:04.800819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a7e99-3590-4771-b029-4ea36e9ee09e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:04.801131 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a7e99-3590-4771-b029-4ea36e9ee09e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:04.801430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a7e99-3590-4771-b029-4ea36e9ee09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:04.819627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6f3a7e99-3590-4771-b029-4ea36e9ee09e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:04.820804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6f3a7e99-3590-4771-b029-4ea36e9ee09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:04.881227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1252/2489] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:04.888939 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096a851-236a-438f-a179-c475d6a720ad None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:04.889276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096a851-236a-438f-a179-c475d6a720ad None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:04.889560 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096a851-236a-438f-a179-c475d6a720ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:04.909184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9096a851-236a-438f-a179-c475d6a720ad None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:04.910262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9096a851-236a-438f-a179-c475d6a720ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:04.919034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1238/2490] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:04.923279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a7e99-3590-4771-b029-4ea36e9ee09e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:04.933067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a7e99-3590-4771-b029-4ea36e9ee09e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:04.938957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:04.939566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:04.939858 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:04.940191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:04.941133 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:04.941450 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:04.943121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:04.958877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1253/2491] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:04.960526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096a851-236a-438f-a179-c475d6a720ad None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:04.970037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096a851-236a-438f-a179-c475d6a720ad None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:04.974836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcc20e-c67f-4a25-a648-c3afe87be0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:04.976053 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcc20e-c67f-4a25-a648-c3afe87be0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:04.976337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcc20e-c67f-4a25-a648-c3afe87be0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:04.976612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcc20e-c67f-4a25-a648-c3afe87be0e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:04.977334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcc20e-c67f-4a25-a648-c3afe87be0e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:04.979310 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcc20e-c67f-4a25-a648-c3afe87be0e1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:05.103867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1239/2492] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:04 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:05.105277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:05.115716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aacf2b-e280-4636-a7f2-b56b20224098 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:05.173350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa4033-9feb-4837-8f3b-3611acdba24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:05.174092 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa4033-9feb-4837-8f3b-3611acdba24d admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.174387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa4033-9feb-4837-8f3b-3611acdba24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.174669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa4033-9feb-4837-8f3b-3611acdba24d admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.175786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa4033-9feb-4837-8f3b-3611acdba24d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:05.177738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faa4033-9feb-4837-8f3b-3611acdba24d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:05.249019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1254/2493] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:05 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:05.253381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcc20e-c67f-4a25-a648-c3afe87be0e1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:05.265189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcc20e-c67f-4a25-a648-c3afe87be0e1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:05.270144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c6cf4-2030-40ea-a7aa-f40ec90d16ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:05.270821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6cf4-2030-40ea-a7aa-f40ec90d16ac admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.271118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6cf4-2030-40ea-a7aa-f40ec90d16ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.271514 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6cf4-2030-40ea-a7aa-f40ec90d16ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.272830 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c6cf4-2030-40ea-a7aa-f40ec90d16ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:05.274614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c6cf4-2030-40ea-a7aa-f40ec90d16ac admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:05.286328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1240/2494] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:05.290713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa4033-9feb-4837-8f3b-3611acdba24d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:05.301922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa4033-9feb-4837-8f3b-3611acdba24d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:05.307039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be76d72-a952-4320-85be-6cc22f99a1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:05.307711 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76d72-a952-4320-85be-6cc22f99a1fa admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.308001 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76d72-a952-4320-85be-6cc22f99a1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.308281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76d72-a952-4320-85be-6cc22f99a1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.309010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be76d72-a952-4320-85be-6cc22f99a1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:05.310657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be76d72-a952-4320-85be-6cc22f99a1fa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:05.321811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1255/2495] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:05.331796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c6cf4-2030-40ea-a7aa-f40ec90d16ac admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:05.342818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c6cf4-2030-40ea-a7aa-f40ec90d16ac admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:05.348538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:05.349523 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.349811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.350119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] PATH_INFO: `/v3/projects/00a59e70e3764eb79d8d096d0307eeb2/users/c9eb085bc0014071b60c2deb59503134/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.365670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a59e70e3764eb79d8d096d0307eeb2, user_id=c9eb085bc0014071b60c2deb59503134, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:05.368472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:05.434198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1241/2496] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:05 2022] PUT /identity/v3/projects/00a59e70e3764eb79d8d096d0307eeb2/users/c9eb085bc0014071b60c2deb59503134/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:05.444713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439279f-afea-4609-9ede-079a1b1fe349 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.445008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439279f-afea-4609-9ede-079a1b1fe349 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.445322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439279f-afea-4609-9ede-079a1b1fe349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.451239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:05.472649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8439279f-afea-4609-9ede-079a1b1fe349 None None] MFA Rules not processed for user `c9eb085bc0014071b60c2deb59503134`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:05.473908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8439279f-afea-4609-9ede-079a1b1fe349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:05.545488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1256/2497] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:05.556228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:05.558266 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:05.583337 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:05.643660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d1558-0e21-4abe-9510-0ebd4ee941bb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:05.650675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:05.651542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.651821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.652189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.661622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:05.663472 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:05.674875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1242/2498] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:05.692875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de2eab-7b25-404e-8256-2401b91c5233 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.692875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de2eab-7b25-404e-8256-2401b91c5233 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.692875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de2eab-7b25-404e-8256-2401b91c5233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.712461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f4de2eab-7b25-404e-8256-2401b91c5233 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:05.712461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f4de2eab-7b25-404e-8256-2401b91c5233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:05.790985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1257/2499] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:43:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:05.805877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:05.807611 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:05.898234 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73128a-f6e2-4716-8314-914fdd83e708 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:05.898322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de2eab-7b25-404e-8256-2401b91c5233 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:05.911093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60477ee-774b-421f-810e-8732c3f32b5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:05.912339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60477ee-774b-421f-810e-8732c3f32b5b service swift] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.912851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60477ee-774b-421f-810e-8732c3f32b5b service swift] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.913386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60477ee-774b-421f-810e-8732c3f32b5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.923768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60477ee-774b-421f-810e-8732c3f32b5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:05.925497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60477ee-774b-421f-810e-8732c3f32b5b service swift] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:05.940374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1243/2500] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:43:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:43:05.982466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de2eab-7b25-404e-8256-2401b91c5233 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:05.988772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:05.990796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:05.991291 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:05.991781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] PATH_INFO: `/v3/users/c9eb085bc0014071b60c2deb59503134` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:05.993132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9eb085bc0014071b60c2deb59503134)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:05.995420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:06.259793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:06.268894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9eb085bc0014071b60c2deb59503134'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:06.284855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9eb085bc0014071b60c2deb59503134'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:06.337264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9eb085bc0014071b60c2deb59503134'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:06.341218 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9eb085bc0014071b60c2deb59503134'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:06.427960 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9eb085bc0014071b60c2deb59503134'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:06.435547 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1258/2501] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:05 2022] DELETE /identity/v3/users/c9eb085bc0014071b60c2deb59503134 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:06.439759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807ba5-9138-4c69-8610-c1c3b06656a9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:06.440300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807ba5-9138-4c69-8610-c1c3b06656a9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:06.440784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807ba5-9138-4c69-8610-c1c3b06656a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:06.465453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-76807ba5-9138-4c69-8610-c1c3b06656a9 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:06.467999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-76807ba5-9138-4c69-8610-c1c3b06656a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:06.576114 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1244/2502] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:06.583992 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:06.586265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:06.679194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c435c3e-2239-4c3a-8847-c5f1db0bd5de admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:06.683268 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bd1e5-7abd-44c4-bb8a-27dc9b4e0c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:06.684175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd1e5-7abd-44c4-bb8a-27dc9b4e0c13 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:06.684464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd1e5-7abd-44c4-bb8a-27dc9b4e0c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:06.684751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bd1e5-7abd-44c4-bb8a-27dc9b4e0c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:06.696281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd1e5-7abd-44c4-bb8a-27dc9b4e0c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:06.697775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bd1e5-7abd-44c4-bb8a-27dc9b4e0c13 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:06.720074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1259/2503] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:07.278743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807ba5-9138-4c69-8610-c1c3b06656a9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:07.280519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-76807ba5-9138-4c69-8610-c1c3b06656a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:07.296142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807ba5-9138-4c69-8610-c1c3b06656a9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:07.304513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:07.305770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:07.306396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:07.306983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] PATH_INFO: `/v3/projects/00a59e70e3764eb79d8d096d0307eeb2` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:07.309339 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a59e70e3764eb79d8d096d0307eeb2)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:07.311852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:07.511565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:07.545295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a59e70e3764eb79d8d096d0307eeb2', 'request_id': 'req-6152599b-7741-43c3-b44b-45ffd12daf87'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:07.556825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a59e70e3764eb79d8d096d0307eeb2', 'request_id': 'req-6152599b-7741-43c3-b44b-45ffd12daf87'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:07.585616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a59e70e3764eb79d8d096d0307eeb2', 'request_id': 'req-6152599b-7741-43c3-b44b-45ffd12daf87'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:07.772885 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] The token cache is being invalidate because project 00a59e70e3764eb79d8d096d0307eeb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:07.773981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:07.776408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-6152599b-7741-43c3-b44b-45ffd12daf87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:07.780342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1245/2504] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:07 2022] DELETE /identity/v3/projects/00a59e70e3764eb79d8d096d0307eeb2 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:07.786012 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff30568-e174-47f0-86d2-c7c805fff254 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:07.788213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff30568-e174-47f0-86d2-c7c805fff254 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:07.789553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff30568-e174-47f0-86d2-c7c805fff254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:07.824754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0ff30568-e174-47f0-86d2-c7c805fff254 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:07.827824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0ff30568-e174-47f0-86d2-c7c805fff254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:07.947842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1260/2505] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:07.956430 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325097-20a7-45d6-92f8-6d3efc77d1f9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:07.956727 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325097-20a7-45d6-92f8-6d3efc77d1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:07.957013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325097-20a7-45d6-92f8-6d3efc77d1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:07.979648 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-67325097-20a7-45d6-92f8-6d3efc77d1f9 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:07.981209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-67325097-20a7-45d6-92f8-6d3efc77d1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:07.993321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1246/2506] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:07.994031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff30568-e174-47f0-86d2-c7c805fff254 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:08.005371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff30568-e174-47f0-86d2-c7c805fff254 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:08.010259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:08.010931 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:08.011222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:08.011504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:08.012276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:08.012563 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:08.014232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:08.029224 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1261/2507] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:43:08.032684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67325097-20a7-45d6-92f8-6d3efc77d1f9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:08.043368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67325097-20a7-45d6-92f8-6d3efc77d1f9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:08.048638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca793128-5f5e-4eda-902f-09905c12174e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:08.049652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca793128-5f5e-4eda-902f-09905c12174e admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:08.050168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca793128-5f5e-4eda-902f-09905c12174e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:08.050832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca793128-5f5e-4eda-902f-09905c12174e admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:08.051754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca793128-5f5e-4eda-902f-09905c12174e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:08.053959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca793128-5f5e-4eda-902f-09905c12174e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:08.156356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1247/2508] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:08 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:08.164007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:08.172084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d79103-1fb2-4a1b-aa17-cfeb3b07d057 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:08.227548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:08.242608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a75091-51a4-48b2-bf56-1ddae6202644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:08.243776 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a75091-51a4-48b2-bf56-1ddae6202644 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:08.244631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a75091-51a4-48b2-bf56-1ddae6202644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:08.245028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a75091-51a4-48b2-bf56-1ddae6202644 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:08.245816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a75091-51a4-48b2-bf56-1ddae6202644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:08.247841 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a75091-51a4-48b2-bf56-1ddae6202644 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:08.342849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1262/2509] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:08 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:08.346732 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca793128-5f5e-4eda-902f-09905c12174e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:08.361900 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca793128-5f5e-4eda-902f-09905c12174e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:08.370926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63069258-27a0-4704-9ec6-c81d04ef1b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:08.375664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069258-27a0-4704-9ec6-c81d04ef1b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:08.375664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069258-27a0-4704-9ec6-c81d04ef1b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:08.375664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63069258-27a0-4704-9ec6-c81d04ef1b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:08.375664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63069258-27a0-4704-9ec6-c81d04ef1b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:08.375664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63069258-27a0-4704-9ec6-c81d04ef1b1c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:08.388145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1248/2510] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:08.392401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a75091-51a4-48b2-bf56-1ddae6202644 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:08.404647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a75091-51a4-48b2-bf56-1ddae6202644 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:08.409575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a936c99-3233-4fba-87b5-e8572f5eeae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:08.410275 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a936c99-3233-4fba-87b5-e8572f5eeae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:08.410562 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a936c99-3233-4fba-87b5-e8572f5eeae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:08.410844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a936c99-3233-4fba-87b5-e8572f5eeae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:08.411609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a936c99-3233-4fba-87b5-e8572f5eeae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:08.413379 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a936c99-3233-4fba-87b5-e8572f5eeae6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:08.427793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1263/2511] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:08.428621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63069258-27a0-4704-9ec6-c81d04ef1b1c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:08.439551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63069258-27a0-4704-9ec6-c81d04ef1b1c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:08.445370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02203b6c-cea6-4790-bfd7-645ea3abc22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:08.446268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02203b6c-cea6-4790-bfd7-645ea3abc22c admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:08.446554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02203b6c-cea6-4790-bfd7-645ea3abc22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:08.446836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02203b6c-cea6-4790-bfd7-645ea3abc22c admin admin] PATH_INFO: `/v3/projects/875e84ace2ce42c8bab67f0163a45832/users/91a1ed11a56048ae931fdb4f4f774efb/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:08.462789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02203b6c-cea6-4790-bfd7-645ea3abc22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=875e84ace2ce42c8bab67f0163a45832, user_id=91a1ed11a56048ae931fdb4f4f774efb, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:08.465757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02203b6c-cea6-4790-bfd7-645ea3abc22c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:08.552794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1249/2512] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:08 2022] PUT /identity/v3/projects/875e84ace2ce42c8bab67f0163a45832/users/91a1ed11a56048ae931fdb4f4f774efb/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:08.568245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf19b22-5f3e-4a3a-9763-f7ed7b5243bf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:08.569728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf19b22-5f3e-4a3a-9763-f7ed7b5243bf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:08.570197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf19b22-5f3e-4a3a-9763-f7ed7b5243bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:08.602515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fbf19b22-5f3e-4a3a-9763-f7ed7b5243bf None None] MFA Rules not processed for user `91a1ed11a56048ae931fdb4f4f774efb`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:08.603949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fbf19b22-5f3e-4a3a-9763-f7ed7b5243bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:08.696535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1264/2513] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:08.701084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02203b6c-cea6-4790-bfd7-645ea3abc22c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:08.795706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02203b6c-cea6-4790-bfd7-645ea3abc22c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:08.800548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4a6d9-f300-4fc5-a1c0-49649f072548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:08.801350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4a6d9-f300-4fc5-a1c0-49649f072548 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:08.801633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4a6d9-f300-4fc5-a1c0-49649f072548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:08.801932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4a6d9-f300-4fc5-a1c0-49649f072548 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:08.806260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4a6d9-f300-4fc5-a1c0-49649f072548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:08.806260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4a6d9-f300-4fc5-a1c0-49649f072548 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:08.928304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1250/2514] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:08 2022] POST /identity/v3/projects => generated 369 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:08.932317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf19b22-5f3e-4a3a-9763-f7ed7b5243bf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:08.945371 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf19b22-5f3e-4a3a-9763-f7ed7b5243bf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:09.036607 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f5751-d598-4118-bcf8-f0c571fa8111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:09.037428 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f5751-d598-4118-bcf8-f0c571fa8111 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:09.038258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f5751-d598-4118-bcf8-f0c571fa8111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:09.038546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f5751-d598-4118-bcf8-f0c571fa8111 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:09.039254 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f5751-d598-4118-bcf8-f0c571fa8111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:09.041403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f5751-d598-4118-bcf8-f0c571fa8111 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:09.129456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1265/2515] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:08 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:09.130483 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4a6d9-f300-4fc5-a1c0-49649f072548 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:09.141475 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4a6d9-f300-4fc5-a1c0-49649f072548 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:09.146257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f924522-f1f5-4fa0-b51b-b3180f4cab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:09.147484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f924522-f1f5-4fa0-b51b-b3180f4cab67 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:09.147774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f924522-f1f5-4fa0-b51b-b3180f4cab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:09.148049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f924522-f1f5-4fa0-b51b-b3180f4cab67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:09.148772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f924522-f1f5-4fa0-b51b-b3180f4cab67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:09.150432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f924522-f1f5-4fa0-b51b-b3180f4cab67 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:09.167069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1251/2516] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:09.167611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f5751-d598-4118-bcf8-f0c571fa8111 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:09.179911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f5751-d598-4118-bcf8-f0c571fa8111 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:09.197895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b5d13-445a-4f20-93ce-a1f6896dee4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:09.198835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b5d13-445a-4f20-93ce-a1f6896dee4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:09.199136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b5d13-445a-4f20-93ce-a1f6896dee4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:09.199420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b5d13-445a-4f20-93ce-a1f6896dee4b admin admin] PATH_INFO: `/v3/projects/a5b8c34082584124bf83315f26a6acde/users/b6ce73e8d65e4670be422d3be30cc2cc/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:09.218271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b5d13-445a-4f20-93ce-a1f6896dee4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b8c34082584124bf83315f26a6acde, user_id=b6ce73e8d65e4670be422d3be30cc2cc, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:09.220782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b5d13-445a-4f20-93ce-a1f6896dee4b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:09.309679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1266/2517] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:09 2022] PUT /identity/v3/projects/a5b8c34082584124bf83315f26a6acde/users/b6ce73e8d65e4670be422d3be30cc2cc/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:09.313788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f924522-f1f5-4fa0-b51b-b3180f4cab67 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:09.325414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f924522-f1f5-4fa0-b51b-b3180f4cab67 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:09.354640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:09.399346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44b021-9e94-42aa-b999-eed9c349098e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:09.400131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44b021-9e94-42aa-b999-eed9c349098e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:09.400491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44b021-9e94-42aa-b999-eed9c349098e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:09.400840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44b021-9e94-42aa-b999-eed9c349098e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:09.401685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44b021-9e94-42aa-b999-eed9c349098e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:09.403853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44b021-9e94-42aa-b999-eed9c349098e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:09.415186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1252/2518] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:09.419115 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b5d13-445a-4f20-93ce-a1f6896dee4b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:09.430662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b5d13-445a-4f20-93ce-a1f6896dee4b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:09.435703 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:09.436645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:09.436949 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:09.437230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] PATH_INFO: `/v3/projects/a5b8c34082584124bf83315f26a6acde/users/b6ce73e8d65e4670be422d3be30cc2cc/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:09.439249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b8c34082584124bf83315f26a6acde, user_id=b6ce73e8d65e4670be422d3be30cc2cc, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:09.441988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:09.454049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1267/2519] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:09 2022] PUT /identity/v3/projects/a5b8c34082584124bf83315f26a6acde/users/b6ce73e8d65e4670be422d3be30cc2cc/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:09.464619 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b047-5be3-4488-ac25-cf6c20c5e169 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:09.470021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b047-5be3-4488-ac25-cf6c20c5e169 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:09.470021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8b047-5be3-4488-ac25-cf6c20c5e169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:09.494667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c8c8b047-5be3-4488-ac25-cf6c20c5e169 None None] MFA Rules not processed for user `b6ce73e8d65e4670be422d3be30cc2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:09.496212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c8c8b047-5be3-4488-ac25-cf6c20c5e169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:09.575366 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1253/2520] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:09 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:09.586793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:09.588462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:09.598432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:09.649802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd2e00-94d1-4bc0-9b54-7d51b62c7403 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:09.653656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00434bb4-55a9-4788-ab12-3fe72fad48f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:09.654301 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00434bb4-55a9-4788-ab12-3fe72fad48f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:09.654584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00434bb4-55a9-4788-ab12-3fe72fad48f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:09.654861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00434bb4-55a9-4788-ab12-3fe72fad48f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:09.671023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00434bb4-55a9-4788-ab12-3fe72fad48f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:09.673167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00434bb4-55a9-4788-ab12-3fe72fad48f8 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:09.687202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1268/2521] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:09 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:09.798123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8b047-5be3-4488-ac25-cf6c20c5e169 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:09.810222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8b047-5be3-4488-ac25-cf6c20c5e169 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:09.823198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91578e-2c49-4f9e-bbf1-520e29ef435d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:09.823198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91578e-2c49-4f9e-bbf1-520e29ef435d service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:09.823198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91578e-2c49-4f9e-bbf1-520e29ef435d service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:09.823198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91578e-2c49-4f9e-bbf1-520e29ef435d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:09.932250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91578e-2c49-4f9e-bbf1-520e29ef435d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:09.933760 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91578e-2c49-4f9e-bbf1-520e29ef435d service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:09.945936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1254/2522] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:15.593173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00434bb4-55a9-4788-ab12-3fe72fad48f8 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:15.655235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00434bb4-55a9-4788-ab12-3fe72fad48f8 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:15.659851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:15.661697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:15.661697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:15.661697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] PATH_INFO: `/v3/users/91a1ed11a56048ae931fdb4f4f774efb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:15.662784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a1ed11a56048ae931fdb4f4f774efb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:15.665515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:15.829512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a1ed11a56048ae931fdb4f4f774efb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:15.836908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a1ed11a56048ae931fdb4f4f774efb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:15.848335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a1ed11a56048ae931fdb4f4f774efb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:15.849287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a1ed11a56048ae931fdb4f4f774efb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:15.899237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a1ed11a56048ae931fdb4f4f774efb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:15.909961 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1269/2523] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:15 2022] DELETE /identity/v3/users/91a1ed11a56048ae931fdb4f4f774efb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:15.914641 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4fe9f-6da8-4ae6-b8f0-8ce49a3ac8ac None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:15.915126 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4fe9f-6da8-4ae6-b8f0-8ce49a3ac8ac None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:15.915249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4fe9f-6da8-4ae6-b8f0-8ce49a3ac8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:15.933500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-96c4fe9f-6da8-4ae6-b8f0-8ce49a3ac8ac None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:15.934699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-96c4fe9f-6da8-4ae6-b8f0-8ce49a3ac8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:15.978647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:15.993387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1255/2524] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:16.003208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:16.004662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:16.038679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:16.063162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f8916-d0ce-4d93-86ec-8f386901484d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:16.066827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff44b6-b34d-4efe-965a-4b692d348dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:16.067466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff44b6-b34d-4efe-965a-4b692d348dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:16.067749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff44b6-b34d-4efe-965a-4b692d348dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:16.068026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff44b6-b34d-4efe-965a-4b692d348dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:16.078525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff44b6-b34d-4efe-965a-4b692d348dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:16.080008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff44b6-b34d-4efe-965a-4b692d348dc1 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:16.090422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1270/2525] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:16.577453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4fe9f-6da8-4ae6-b8f0-8ce49a3ac8ac None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:16.578983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-96c4fe9f-6da8-4ae6-b8f0-8ce49a3ac8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:16.591277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4fe9f-6da8-4ae6-b8f0-8ce49a3ac8ac None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:16.600949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:16.601788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:16.602074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:16.602347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] PATH_INFO: `/v3/projects/875e84ace2ce42c8bab67f0163a45832` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:16.603540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] RBAC: Authorizing `identity:delete_project(project_id=875e84ace2ce42c8bab67f0163a45832)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:16.605568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:16.713818 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '875e84ace2ce42c8bab67f0163a45832', 'request_id': 'req-5e1b31db-df43-46dc-839a-fdb781b2695e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:16.720196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '875e84ace2ce42c8bab67f0163a45832', 'request_id': 'req-5e1b31db-df43-46dc-839a-fdb781b2695e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:16.725789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '875e84ace2ce42c8bab67f0163a45832', 'request_id': 'req-5e1b31db-df43-46dc-839a-fdb781b2695e'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:16.807155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] The token cache is being invalidate because project 875e84ace2ce42c8bab67f0163a45832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:16.807468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:16.808079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:16.809700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1256/2526] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:16 2022] DELETE /identity/v3/projects/875e84ace2ce42c8bab67f0163a45832 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:16.813810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff44b6-b34d-4efe-965a-4b692d348dc1 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:16.823987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff44b6-b34d-4efe-965a-4b692d348dc1 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:16.878433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:16.879225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:16.879507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:16.879781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] PATH_INFO: `/v3/users/b6ce73e8d65e4670be422d3be30cc2cc` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:16.880937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ce73e8d65e4670be422d3be30cc2cc)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:16.883399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:17.076743 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ce73e8d65e4670be422d3be30cc2cc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.083968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ce73e8d65e4670be422d3be30cc2cc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.095847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ce73e8d65e4670be422d3be30cc2cc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.096551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ce73e8d65e4670be422d3be30cc2cc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.158832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3339a16e-476d-4dbf-b42f-4f15fa97108f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ce73e8d65e4670be422d3be30cc2cc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.165374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1271/2527] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:16 2022] DELETE /identity/v3/users/b6ce73e8d65e4670be422d3be30cc2cc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:17.578609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:17.581147 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:17.594690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b31db-df43-46dc-839a-fdb781b2695e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:17.653008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:17.653816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:17.654100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:17.654377 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] PATH_INFO: `/v3/projects/a5b8c34082584124bf83315f26a6acde` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:17.655555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b8c34082584124bf83315f26a6acde)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:17.657500 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:17.794391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b8c34082584124bf83315f26a6acde', 'request_id': 'req-81a653a6-5e8d-4cd2-94e0-1642360f53cd'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.800512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b8c34082584124bf83315f26a6acde', 'request_id': 'req-81a653a6-5e8d-4cd2-94e0-1642360f53cd'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.806083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b8c34082584124bf83315f26a6acde', 'request_id': 'req-81a653a6-5e8d-4cd2-94e0-1642360f53cd'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.878318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] The token cache is being invalidate because project a5b8c34082584124bf83315f26a6acde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:17.878606 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.879220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-81a653a6-5e8d-4cd2-94e0-1642360f53cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:17.880749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1257/2528] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:17 2022] DELETE /identity/v3/projects/a5b8c34082584124bf83315f26a6acde => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:17.885693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6475cc-069d-4bf1-881f-cfbd13d3e217 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:17.889173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6475cc-069d-4bf1-881f-cfbd13d3e217 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:17.889173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6475cc-069d-4bf1-881f-cfbd13d3e217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:17.904282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3a6475cc-069d-4bf1-881f-cfbd13d3e217 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:17.905444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a6475cc-069d-4bf1-881f-cfbd13d3e217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:17.953660 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:17.972281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1272/2529] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:17.981689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ec26c-cb7f-47a0-bb55-cacb7a936820 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:17.981978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ec26c-cb7f-47a0-bb55-cacb7a936820 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:17.982304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ec26c-cb7f-47a0-bb55-cacb7a936820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:17.996173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:18.003771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4c0ec26c-cb7f-47a0-bb55-cacb7a936820 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:18.005287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4c0ec26c-cb7f-47a0-bb55-cacb7a936820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:18.020069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1258/2530] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:18.026844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6475cc-069d-4bf1-881f-cfbd13d3e217 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:18.038396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6475cc-069d-4bf1-881f-cfbd13d3e217 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:18.043426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:18.044084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.044432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.044697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.045468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:18.045774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:18.057624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:18.059193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1273/2531] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:18.063599 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ec26c-cb7f-47a0-bb55-cacb7a936820 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:18.073302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ec26c-cb7f-47a0-bb55-cacb7a936820 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:18.078449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eedccc-b7f3-48da-af0c-cd89ef7f2c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:18.079097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eedccc-b7f3-48da-af0c-cd89ef7f2c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.079407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eedccc-b7f3-48da-af0c-cd89ef7f2c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.079856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eedccc-b7f3-48da-af0c-cd89ef7f2c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.080569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eedccc-b7f3-48da-af0c-cd89ef7f2c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:18.082446 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eedccc-b7f3-48da-af0c-cd89ef7f2c82 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:18.163800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1259/2532] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:18 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:18.167896 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:18.178223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4e359-1984-44eb-9232-4014ff99447e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:18.237779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed77af-95bb-48ba-98c7-dec3fbda917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:18.238458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed77af-95bb-48ba-98c7-dec3fbda917d admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.238747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed77af-95bb-48ba-98c7-dec3fbda917d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.239031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed77af-95bb-48ba-98c7-dec3fbda917d admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.239715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed77af-95bb-48ba-98c7-dec3fbda917d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:18.241667 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed77af-95bb-48ba-98c7-dec3fbda917d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:18.317295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1274/2533] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:18 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:18.321827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eedccc-b7f3-48da-af0c-cd89ef7f2c82 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:18.333959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eedccc-b7f3-48da-af0c-cd89ef7f2c82 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:18.339602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0a5c-51f0-457a-89ba-7e9ebe0b7601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:18.340340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0a5c-51f0-457a-89ba-7e9ebe0b7601 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.340733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0a5c-51f0-457a-89ba-7e9ebe0b7601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.341082 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0a5c-51f0-457a-89ba-7e9ebe0b7601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.341856 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0a5c-51f0-457a-89ba-7e9ebe0b7601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:18.343464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0a5c-51f0-457a-89ba-7e9ebe0b7601 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:18.353945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1260/2534] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:18.358271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed77af-95bb-48ba-98c7-dec3fbda917d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:18.369382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed77af-95bb-48ba-98c7-dec3fbda917d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:18.374455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2db1-cf3f-4908-8cb9-4a04738bd46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:18.375157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2db1-cf3f-4908-8cb9-4a04738bd46e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.375446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2db1-cf3f-4908-8cb9-4a04738bd46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.375939 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2db1-cf3f-4908-8cb9-4a04738bd46e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.376671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af2db1-cf3f-4908-8cb9-4a04738bd46e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:18.379173 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af2db1-cf3f-4908-8cb9-4a04738bd46e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:18.393335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1275/2535] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:18.395306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0a5c-51f0-457a-89ba-7e9ebe0b7601 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:18.407065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0a5c-51f0-457a-89ba-7e9ebe0b7601 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:18.411903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f88da-4492-40cd-804e-df2db634b88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:18.412930 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f88da-4492-40cd-804e-df2db634b88b admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.413264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f88da-4492-40cd-804e-df2db634b88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.413686 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f88da-4492-40cd-804e-df2db634b88b admin admin] PATH_INFO: `/v3/projects/5882cebc5bd947bfb369949b3af13f24/users/d4930363fd9f4f83af082a1230665476/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.438624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f88da-4492-40cd-804e-df2db634b88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5882cebc5bd947bfb369949b3af13f24, user_id=d4930363fd9f4f83af082a1230665476, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:18.438624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f88da-4492-40cd-804e-df2db634b88b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:18.522603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1261/2536] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:18 2022] PUT /identity/v3/projects/5882cebc5bd947bfb369949b3af13f24/users/d4930363fd9f4f83af082a1230665476/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:18.535837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b13f-35d3-4f10-9b3f-6c598488a1bc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.535837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b13f-35d3-4f10-9b3f-6c598488a1bc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.535837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b13f-35d3-4f10-9b3f-6c598488a1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.578895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dbb2b13f-35d3-4f10-9b3f-6c598488a1bc None None] MFA Rules not processed for user `d4930363fd9f4f83af082a1230665476`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:18.580274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dbb2b13f-35d3-4f10-9b3f-6c598488a1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:18.654061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1276/2537] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:18.658592 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f88da-4492-40cd-804e-df2db634b88b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:18.722512 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f88da-4492-40cd-804e-df2db634b88b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:18.727205 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83948d7f-d629-4717-96ae-f3da65cee0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:18.727880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83948d7f-d629-4717-96ae-f3da65cee0ef admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.728168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83948d7f-d629-4717-96ae-f3da65cee0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.728442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83948d7f-d629-4717-96ae-f3da65cee0ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.729168 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83948d7f-d629-4717-96ae-f3da65cee0ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:18.731059 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83948d7f-d629-4717-96ae-f3da65cee0ef admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:18.838155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1262/2538] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:18 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:18.842032 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b13f-35d3-4f10-9b3f-6c598488a1bc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:18.853404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b13f-35d3-4f10-9b3f-6c598488a1bc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:18.896608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:18.909822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529534f-8682-4f1a-abd1-c195f2d748cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:18.910567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529534f-8682-4f1a-abd1-c195f2d748cb admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:18.910865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529534f-8682-4f1a-abd1-c195f2d748cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:18.911153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529534f-8682-4f1a-abd1-c195f2d748cb admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:18.911847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529534f-8682-4f1a-abd1-c195f2d748cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:18.914451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529534f-8682-4f1a-abd1-c195f2d748cb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:18.999753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1277/2539] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:18 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:19.003731 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83948d7f-d629-4717-96ae-f3da65cee0ef admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:19.014193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83948d7f-d629-4717-96ae-f3da65cee0ef admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:19.018787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fdce-8619-4b09-a1a3-ff57d8213721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:19.019437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fdce-8619-4b09-a1a3-ff57d8213721 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:19.019718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fdce-8619-4b09-a1a3-ff57d8213721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:19.019994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121fdce-8619-4b09-a1a3-ff57d8213721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:19.020709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121fdce-8619-4b09-a1a3-ff57d8213721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:19.022365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121fdce-8619-4b09-a1a3-ff57d8213721 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:19.032643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1263/2540] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:19.036843 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529534f-8682-4f1a-abd1-c195f2d748cb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:19.047255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529534f-8682-4f1a-abd1-c195f2d748cb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:19.052129 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fefeda-70e9-47d9-8119-827dc779da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:19.057365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fefeda-70e9-47d9-8119-827dc779da5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:19.057365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fefeda-70e9-47d9-8119-827dc779da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:19.057365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fefeda-70e9-47d9-8119-827dc779da5c admin admin] PATH_INFO: `/v3/projects/8dae182359a046049e64bcbcf794d35c/users/608a15b0ae1b40968e2ae82e5a5bcda1/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:19.068916 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fefeda-70e9-47d9-8119-827dc779da5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dae182359a046049e64bcbcf794d35c, user_id=608a15b0ae1b40968e2ae82e5a5bcda1, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:19.071595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fefeda-70e9-47d9-8119-827dc779da5c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:19.141312 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1278/2541] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:19 2022] PUT /identity/v3/projects/8dae182359a046049e64bcbcf794d35c/users/608a15b0ae1b40968e2ae82e5a5bcda1/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:19.145378 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fdce-8619-4b09-a1a3-ff57d8213721 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:19.155578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121fdce-8619-4b09-a1a3-ff57d8213721 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:19.210040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e66d9d-dd3f-4d24-b983-f4964ffea8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:19.210772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66d9d-dd3f-4d24-b983-f4964ffea8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:19.211051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66d9d-dd3f-4d24-b983-f4964ffea8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:19.211325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66d9d-dd3f-4d24-b983-f4964ffea8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:19.212093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e66d9d-dd3f-4d24-b983-f4964ffea8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:19.213722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e66d9d-dd3f-4d24-b983-f4964ffea8f9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:19.223830 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1264/2542] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:19.228596 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fefeda-70e9-47d9-8119-827dc779da5c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:19.238910 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fefeda-70e9-47d9-8119-827dc779da5c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:19.243531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:19.244396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:19.244943 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:19.245278 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] PATH_INFO: `/v3/projects/8dae182359a046049e64bcbcf794d35c/users/608a15b0ae1b40968e2ae82e5a5bcda1/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:19.247270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dae182359a046049e64bcbcf794d35c, user_id=608a15b0ae1b40968e2ae82e5a5bcda1, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:19.249983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:19.260174 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1279/2543] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:19 2022] PUT /identity/v3/projects/8dae182359a046049e64bcbcf794d35c/users/608a15b0ae1b40968e2ae82e5a5bcda1/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:19.269927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d91e7-a01b-41ac-9a1e-4a7b24baeec2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:19.270227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d91e7-a01b-41ac-9a1e-4a7b24baeec2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:19.270508 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d91e7-a01b-41ac-9a1e-4a7b24baeec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:19.291665 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:19.300155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-690d91e7-a01b-41ac-9a1e-4a7b24baeec2 None None] MFA Rules not processed for user `608a15b0ae1b40968e2ae82e5a5bcda1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:19.301397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-690d91e7-a01b-41ac-9a1e-4a7b24baeec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:19.372041 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1265/2544] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:19.392306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:19.394139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:19.452132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85de792-dd31-4d8a-817e-8c576e41f9f4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:19.455823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9f68a-ef22-4969-ba79-36e846430402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:19.456464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9f68a-ef22-4969-ba79-36e846430402 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:19.456753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9f68a-ef22-4969-ba79-36e846430402 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:19.457043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9f68a-ef22-4969-ba79-36e846430402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:19.466683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9f68a-ef22-4969-ba79-36e846430402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:19.468120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9f68a-ef22-4969-ba79-36e846430402 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:19.478209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1280/2545] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:19.507289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d91e7-a01b-41ac-9a1e-4a7b24baeec2 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:19.568301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d91e7-a01b-41ac-9a1e-4a7b24baeec2 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:19.573449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:19.575331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:19.575331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:19.575331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] PATH_INFO: `/v3/users/d4930363fd9f4f83af082a1230665476` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:19.576014 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4930363fd9f4f83af082a1230665476)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:19.577983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:19.801420 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4930363fd9f4f83af082a1230665476'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:19.820386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4930363fd9f4f83af082a1230665476'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:19.821240 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4930363fd9f4f83af082a1230665476'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:19.821908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4930363fd9f4f83af082a1230665476'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:19.888397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4930363fd9f4f83af082a1230665476'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:19.902393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1266/2546] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:19 2022] DELETE /identity/v3/users/d4930363fd9f4f83af082a1230665476 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:19.909303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d988-e528-4cdd-9107-89cd6ca99f5a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:19.909850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d988-e528-4cdd-9107-89cd6ca99f5a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:19.909850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69d988-e528-4cdd-9107-89cd6ca99f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:19.927489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ee69d988-e528-4cdd-9107-89cd6ca99f5a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:19.929238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ee69d988-e528-4cdd-9107-89cd6ca99f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:19.989054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1281/2547] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:20.000124 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:20.001613 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:20.018253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:20.059008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecd350-fc86-475d-befa-05ba762eda32 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:20.062609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6fe4e-7c34-4b6c-8a3c-5f732369ef72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:20.063251 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6fe4e-7c34-4b6c-8a3c-5f732369ef72 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:20.063535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6fe4e-7c34-4b6c-8a3c-5f732369ef72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:20.063809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6fe4e-7c34-4b6c-8a3c-5f732369ef72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:20.073812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6fe4e-7c34-4b6c-8a3c-5f732369ef72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:20.075342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6fe4e-7c34-4b6c-8a3c-5f732369ef72 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:20.094262 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1267/2548] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:20.497797 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69d988-e528-4cdd-9107-89cd6ca99f5a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:20.498828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ee69d988-e528-4cdd-9107-89cd6ca99f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:20.512583 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69d988-e528-4cdd-9107-89cd6ca99f5a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:20.517970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:20.519315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:20.519315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:20.519522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] PATH_INFO: `/v3/projects/5882cebc5bd947bfb369949b3af13f24` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:20.520747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5882cebc5bd947bfb369949b3af13f24)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:20.525928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:20.659694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:20.668405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5882cebc5bd947bfb369949b3af13f24', 'request_id': 'req-0186f290-1e54-448c-bdc8-8f71ae57c644'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:20.675142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5882cebc5bd947bfb369949b3af13f24', 'request_id': 'req-0186f290-1e54-448c-bdc8-8f71ae57c644'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:20.680985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5882cebc5bd947bfb369949b3af13f24', 'request_id': 'req-0186f290-1e54-448c-bdc8-8f71ae57c644'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:20.764112 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] The token cache is being invalidate because project 5882cebc5bd947bfb369949b3af13f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:20.764404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:20.765189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:20.766765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1282/2549] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:20 2022] DELETE /identity/v3/projects/5882cebc5bd947bfb369949b3af13f24 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:20.770713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6fe4e-7c34-4b6c-8a3c-5f732369ef72 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:20.781079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6fe4e-7c34-4b6c-8a3c-5f732369ef72 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:20.841304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:20.842172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:20.842459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:20.842735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] PATH_INFO: `/v3/users/608a15b0ae1b40968e2ae82e5a5bcda1` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:20.848790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=608a15b0ae1b40968e2ae82e5a5bcda1)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:20.851406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:21.074144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608a15b0ae1b40968e2ae82e5a5bcda1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.086486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608a15b0ae1b40968e2ae82e5a5bcda1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.087139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608a15b0ae1b40968e2ae82e5a5bcda1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.087785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608a15b0ae1b40968e2ae82e5a5bcda1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.186751 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-159a7844-2c2c-4ff2-8a3b-e68ea4271d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608a15b0ae1b40968e2ae82e5a5bcda1'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.201579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1268/2550] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:20 2022] DELETE /identity/v3/users/608a15b0ae1b40968e2ae82e5a5bcda1 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:21.597887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:21.599403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:21.613232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186f290-1e54-448c-bdc8-8f71ae57c644 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:21.670974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:21.671835 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:21.672122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:21.672403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] PATH_INFO: `/v3/projects/8dae182359a046049e64bcbcf794d35c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:21.673619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dae182359a046049e64bcbcf794d35c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:21.675524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:21.803058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dae182359a046049e64bcbcf794d35c', 'request_id': 'req-972d50da-35fd-407f-b6af-6f3de16ea066'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.809889 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dae182359a046049e64bcbcf794d35c', 'request_id': 'req-972d50da-35fd-407f-b6af-6f3de16ea066'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.815772 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dae182359a046049e64bcbcf794d35c', 'request_id': 'req-972d50da-35fd-407f-b6af-6f3de16ea066'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.904699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] The token cache is being invalidate because project 8dae182359a046049e64bcbcf794d35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:21.905546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.905728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-972d50da-35fd-407f-b6af-6f3de16ea066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:21.907614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1283/2551] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:21 2022] DELETE /identity/v3/projects/8dae182359a046049e64bcbcf794d35c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:21.917734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9587-4b67-4522-91ed-783d3e8000fe None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:21.918191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9587-4b67-4522-91ed-783d3e8000fe None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:21.921747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b9587-4b67-4522-91ed-783d3e8000fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:21.937793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-bc4b9587-4b67-4522-91ed-783d3e8000fe None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:21.939136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bc4b9587-4b67-4522-91ed-783d3e8000fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:21.970305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:22.000253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1269/2552] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:22.007887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa5af5-6925-400e-934a-2bc3b30efffc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.008186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa5af5-6925-400e-934a-2bc3b30efffc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.008507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa5af5-6925-400e-934a-2bc3b30efffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.031775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-70fa5af5-6925-400e-934a-2bc3b30efffc None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:22.032103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-70fa5af5-6925-400e-934a-2bc3b30efffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:22.049152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1284/2553] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:22.052453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b9587-4b67-4522-91ed-783d3e8000fe None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:22.068704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b9587-4b67-4522-91ed-783d3e8000fe None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:22.076128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:22.077340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.077796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.078296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.079488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:22.079941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:22.083277 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:22.099072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1270/2554] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:22.104082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa5af5-6925-400e-934a-2bc3b30efffc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:22.114832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa5af5-6925-400e-934a-2bc3b30efffc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:22.122250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d99f77-83e0-4850-8e18-c7cf1c282e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:22.123228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d99f77-83e0-4850-8e18-c7cf1c282e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.123611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d99f77-83e0-4850-8e18-c7cf1c282e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.124017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d99f77-83e0-4850-8e18-c7cf1c282e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.124814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d99f77-83e0-4850-8e18-c7cf1c282e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:22.126833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d99f77-83e0-4850-8e18-c7cf1c282e52 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:22.250090 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1285/2555] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:22 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:22.251029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:22.264968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b079f-be47-43f9-9a90-2a80dce00320 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:22.326049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e618e4-113f-4ff6-a593-806d41ae68c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:22.326814 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e618e4-113f-4ff6-a593-806d41ae68c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.327101 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e618e4-113f-4ff6-a593-806d41ae68c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.327381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e618e4-113f-4ff6-a593-806d41ae68c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.328067 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e618e4-113f-4ff6-a593-806d41ae68c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:22.330023 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e618e4-113f-4ff6-a593-806d41ae68c0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:22.424688 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1271/2556] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:22 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:22.426686 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d99f77-83e0-4850-8e18-c7cf1c282e52 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:22.436928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d99f77-83e0-4850-8e18-c7cf1c282e52 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:22.441622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bd93f-f387-4a64-a936-f51781babe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:22.442392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bd93f-f387-4a64-a936-f51781babe4a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.442840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bd93f-f387-4a64-a936-f51781babe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.443123 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701bd93f-f387-4a64-a936-f51781babe4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.443837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bd93f-f387-4a64-a936-f51781babe4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:22.445458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701bd93f-f387-4a64-a936-f51781babe4a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:22.456391 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1286/2557] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:22.461846 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e618e4-113f-4ff6-a593-806d41ae68c0 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:22.471752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e618e4-113f-4ff6-a593-806d41ae68c0 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:22.477094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8c0bb-45db-4ea6-a2b7-8afa4b969785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:22.477778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8c0bb-45db-4ea6-a2b7-8afa4b969785 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.478064 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8c0bb-45db-4ea6-a2b7-8afa4b969785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.478376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8c0bb-45db-4ea6-a2b7-8afa4b969785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.479107 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8c0bb-45db-4ea6-a2b7-8afa4b969785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:22.480717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8c0bb-45db-4ea6-a2b7-8afa4b969785 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:22.500945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1272/2558] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:22.501427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bd93f-f387-4a64-a936-f51781babe4a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:22.505474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701bd93f-f387-4a64-a936-f51781babe4a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:22.511255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:22.512228 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.512640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.512920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] PATH_INFO: `/v3/projects/1cc8bffa64ae4b788435d1ea8711b257/users/f449fe23f1a545a6a055dfd8c2a5638a/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.525671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:22.529406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc8bffa64ae4b788435d1ea8711b257, user_id=f449fe23f1a545a6a055dfd8c2a5638a, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:22.532043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:22.622307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1287/2559] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:22 2022] PUT /identity/v3/projects/1cc8bffa64ae4b788435d1ea8711b257/users/f449fe23f1a545a6a055dfd8c2a5638a/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:22.633178 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf06680-488e-4144-adeb-5611a4c2392a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.633478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf06680-488e-4144-adeb-5611a4c2392a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.633765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf06680-488e-4144-adeb-5611a4c2392a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.660527 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-aaf06680-488e-4144-adeb-5611a4c2392a None None] MFA Rules not processed for user `f449fe23f1a545a6a055dfd8c2a5638a`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:22.661771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-aaf06680-488e-4144-adeb-5611a4c2392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:22.725507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1273/2560] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:22.736501 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:22.740088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:22.795895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960b485-7ff9-4706-8248-41253b8edd92 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:22.799720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a477a0-09ed-48a8-8d23-821f45660b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:22.800358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a477a0-09ed-48a8-8d23-821f45660b91 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.800642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a477a0-09ed-48a8-8d23-821f45660b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.800919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a477a0-09ed-48a8-8d23-821f45660b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.810519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a477a0-09ed-48a8-8d23-821f45660b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:22.811944 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a477a0-09ed-48a8-8d23-821f45660b91 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:22.821184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1288/2561] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:22.840551 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf06680-488e-4144-adeb-5611a4c2392a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:22.874080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:22.903871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf06680-488e-4144-adeb-5611a4c2392a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:22.908626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:22.909707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:22.909989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:22.910274 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] PATH_INFO: `/v3/users/f449fe23f1a545a6a055dfd8c2a5638a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:22.911489 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f449fe23f1a545a6a055dfd8c2a5638a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:22.913414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:23.098495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f449fe23f1a545a6a055dfd8c2a5638a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:23.110105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f449fe23f1a545a6a055dfd8c2a5638a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:23.110830 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f449fe23f1a545a6a055dfd8c2a5638a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:23.111436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f449fe23f1a545a6a055dfd8c2a5638a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:23.187063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f449fe23f1a545a6a055dfd8c2a5638a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:23.200221 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1274/2562] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:22 2022] DELETE /identity/v3/users/f449fe23f1a545a6a055dfd8c2a5638a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:23.204677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bf8b6-a4f1-432d-bb21-c67ee719a0db None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:23.205317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bf8b6-a4f1-432d-bb21-c67ee719a0db None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:23.205706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bf8b6-a4f1-432d-bb21-c67ee719a0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:23.223643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6a3bf8b6-a4f1-432d-bb21-c67ee719a0db None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:23.225044 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6a3bf8b6-a4f1-432d-bb21-c67ee719a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:23.290100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1289/2563] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:23.299666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:23.301077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:23.359494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174b1f0-b341-45e2-8618-b624b984dc51 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:23.370291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fab58a-562b-40ed-9366-a7d078142465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:23.370935 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab58a-562b-40ed-9366-a7d078142465 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:23.372603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab58a-562b-40ed-9366-a7d078142465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:23.372603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab58a-562b-40ed-9366-a7d078142465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:23.380783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fab58a-562b-40ed-9366-a7d078142465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:23.382272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fab58a-562b-40ed-9366-a7d078142465 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:23.391531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1275/2564] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:23.827016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bf8b6-a4f1-432d-bb21-c67ee719a0db None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:23.828564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6a3bf8b6-a4f1-432d-bb21-c67ee719a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:23.841752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bf8b6-a4f1-432d-bb21-c67ee719a0db None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:23.846720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:23.848096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:23.848381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:23.848678 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] PATH_INFO: `/v3/projects/1cc8bffa64ae4b788435d1ea8711b257` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:23.849866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc8bffa64ae4b788435d1ea8711b257)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:23.852875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:23.973200 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc8bffa64ae4b788435d1ea8711b257', 'request_id': 'req-79769ae1-b828-4514-9856-85d02c192d96'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:23.980730 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc8bffa64ae4b788435d1ea8711b257', 'request_id': 'req-79769ae1-b828-4514-9856-85d02c192d96'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:23.986519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc8bffa64ae4b788435d1ea8711b257', 'request_id': 'req-79769ae1-b828-4514-9856-85d02c192d96'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:24.075695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] The token cache is being invalidate because project 1cc8bffa64ae4b788435d1ea8711b257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:24.079008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:24.079008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79769ae1-b828-4514-9856-85d02c192d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:24.079008 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1290/2565] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:23 2022] DELETE /identity/v3/projects/1cc8bffa64ae4b788435d1ea8711b257 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:24.083487 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab2807-264b-47b0-a549-a9997b54bfcf None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.083697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab2807-264b-47b0-a549-a9997b54bfcf None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.083948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab2807-264b-47b0-a549-a9997b54bfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.103518 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-34ab2807-264b-47b0-a549-a9997b54bfcf None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:24.105486 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-34ab2807-264b-47b0-a549-a9997b54bfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:24.175965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:24.196677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1276/2566] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:24.198804 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd403d-0cb6-4564-a081-e983c44a6977 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.199095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd403d-0cb6-4564-a081-e983c44a6977 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.203783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd403d-0cb6-4564-a081-e983c44a6977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.220443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-85cd403d-0cb6-4564-a081-e983c44a6977 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:24.223177 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-85cd403d-0cb6-4564-a081-e983c44a6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:24.230569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1291/2567] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:24.236530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab2807-264b-47b0-a549-a9997b54bfcf None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:24.247139 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab2807-264b-47b0-a549-a9997b54bfcf None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:24.251934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:24.252611 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.252895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.253202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.254203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:24.254519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:24.256693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:24.269557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1277/2568] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:24.274599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd403d-0cb6-4564-a081-e983c44a6977 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:24.284612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd403d-0cb6-4564-a081-e983c44a6977 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:24.289519 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d37b6-fd60-4d4b-823e-676c42dd74d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:24.290198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d37b6-fd60-4d4b-823e-676c42dd74d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.290474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d37b6-fd60-4d4b-823e-676c42dd74d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.290749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d37b6-fd60-4d4b-823e-676c42dd74d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.291456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d37b6-fd60-4d4b-823e-676c42dd74d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:24.293759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d37b6-fd60-4d4b-823e-676c42dd74d3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:24.303537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:24.419877 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1292/2569] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:24 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:24.421618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:24.446876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df475f85-7114-46fc-9f28-77f02b07fa4c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:24.522319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bd305-2f66-4f38-9444-9dfca8721499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:24.523289 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd305-2f66-4f38-9444-9dfca8721499 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.523571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd305-2f66-4f38-9444-9dfca8721499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.523919 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd305-2f66-4f38-9444-9dfca8721499 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.524610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bd305-2f66-4f38-9444-9dfca8721499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:24.526804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bd305-2f66-4f38-9444-9dfca8721499 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:24.628375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1278/2570] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:24 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:24.633364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d37b6-fd60-4d4b-823e-676c42dd74d3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:24.643553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d37b6-fd60-4d4b-823e-676c42dd74d3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:24.648182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247621e-0290-4f8c-b0b3-185114e40309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:24.648899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247621e-0290-4f8c-b0b3-185114e40309 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.649368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247621e-0290-4f8c-b0b3-185114e40309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.649513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247621e-0290-4f8c-b0b3-185114e40309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.650235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247621e-0290-4f8c-b0b3-185114e40309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:24.651828 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247621e-0290-4f8c-b0b3-185114e40309 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:24.662614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1293/2571] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:24.666651 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bd305-2f66-4f38-9444-9dfca8721499 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:24.676891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bd305-2f66-4f38-9444-9dfca8721499 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:24.684525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc7ddc-9949-4b26-accb-6290a5432c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:24.684525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc7ddc-9949-4b26-accb-6290a5432c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.684525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc7ddc-9949-4b26-accb-6290a5432c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.684525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc7ddc-9949-4b26-accb-6290a5432c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.685017 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc7ddc-9949-4b26-accb-6290a5432c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:24.685622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc7ddc-9949-4b26-accb-6290a5432c9f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:24.696558 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1279/2572] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:24.701128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247621e-0290-4f8c-b0b3-185114e40309 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:24.725308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247621e-0290-4f8c-b0b3-185114e40309 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:24.730756 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:24.737683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.737683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.737683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] PATH_INFO: `/v3/projects/9e4a2936787c417a8f573813e3234743/users/3a0990496d23401fa7c818298b7a45b6/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.750721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e4a2936787c417a8f573813e3234743, user_id=3a0990496d23401fa7c818298b7a45b6, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:24.756128 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:24.866629 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1294/2573] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:24 2022] PUT /identity/v3/projects/9e4a2936787c417a8f573813e3234743/users/3a0990496d23401fa7c818298b7a45b6/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:24.876714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36897ac3-7475-4956-a82e-e6270288cd25 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:24.877068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36897ac3-7475-4956-a82e-e6270288cd25 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:24.877346 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36897ac3-7475-4956-a82e-e6270288cd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:24.911854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-36897ac3-7475-4956-a82e-e6270288cd25 None None] MFA Rules not processed for user `3a0990496d23401fa7c818298b7a45b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:24.913635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-36897ac3-7475-4956-a82e-e6270288cd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:24.981065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1280/2574] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:24.992816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:24.994524 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:25.053708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b6e67-ae54-423a-9c20-aab59cd93efe admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:25.057441 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11f946-6647-4d09-86e3-371f4c7b2705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:25.058083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f946-6647-4d09-86e3-371f4c7b2705 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:25.058370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f946-6647-4d09-86e3-371f4c7b2705 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:25.058649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11f946-6647-4d09-86e3-371f4c7b2705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:25.067193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11f946-6647-4d09-86e3-371f4c7b2705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:25.068884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11f946-6647-4d09-86e3-371f4c7b2705 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:25.083502 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1295/2575] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:30.396541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36897ac3-7475-4956-a82e-e6270288cd25 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:30.448646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:30.465246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36897ac3-7475-4956-a82e-e6270288cd25 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:30.469859 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:30.470949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:30.471228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:30.471535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] PATH_INFO: `/v3/users/3a0990496d23401fa7c818298b7a45b6` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:30.472700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0990496d23401fa7c818298b7a45b6)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:30.475591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:30.662996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0990496d23401fa7c818298b7a45b6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:30.680925 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0990496d23401fa7c818298b7a45b6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:30.681724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0990496d23401fa7c818298b7a45b6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:30.682481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0990496d23401fa7c818298b7a45b6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:30.771536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0990496d23401fa7c818298b7a45b6'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:30.788058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1281/2576] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:30 2022] DELETE /identity/v3/users/3a0990496d23401fa7c818298b7a45b6 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:30.792669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805fcb6-fdb4-4f8b-8f4b-57e6644c1638 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:30.793555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805fcb6-fdb4-4f8b-8f4b-57e6644c1638 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:30.794350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805fcb6-fdb4-4f8b-8f4b-57e6644c1638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:30.814599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c805fcb6-fdb4-4f8b-8f4b-57e6644c1638 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:30.815808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c805fcb6-fdb4-4f8b-8f4b-57e6644c1638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:30.879222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1296/2577] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:30.889577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:30.891051 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:30.961840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec87c88-ff40-4efa-af8f-9c3c6605938f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:30.965704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558811d-42e5-48c0-9bec-2e2d51fae36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:30.966352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558811d-42e5-48c0-9bec-2e2d51fae36b service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:30.966635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558811d-42e5-48c0-9bec-2e2d51fae36b service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:30.969179 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558811d-42e5-48c0-9bec-2e2d51fae36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:30.976920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558811d-42e5-48c0-9bec-2e2d51fae36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:30.978389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558811d-42e5-48c0-9bec-2e2d51fae36b service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:30.991184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1282/2578] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:31.452761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805fcb6-fdb4-4f8b-8f4b-57e6644c1638 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:31.454096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c805fcb6-fdb4-4f8b-8f4b-57e6644c1638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:31.467370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805fcb6-fdb4-4f8b-8f4b-57e6644c1638 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:31.472138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:31.473323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:31.473323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:31.479535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] PATH_INFO: `/v3/projects/9e4a2936787c417a8f573813e3234743` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:31.479535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e4a2936787c417a8f573813e3234743)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:31.479535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:31.593529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e4a2936787c417a8f573813e3234743', 'request_id': 'req-5bdff660-b554-4099-b419-44f27091bfef'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:31.595362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:31.600364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e4a2936787c417a8f573813e3234743', 'request_id': 'req-5bdff660-b554-4099-b419-44f27091bfef'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:31.605969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e4a2936787c417a8f573813e3234743', 'request_id': 'req-5bdff660-b554-4099-b419-44f27091bfef'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:31.716569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] The token cache is being invalidate because project 9e4a2936787c417a8f573813e3234743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:31.716901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:31.717701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bdff660-b554-4099-b419-44f27091bfef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:31.719369 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1297/2579] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:31 2022] DELETE /identity/v3/projects/9e4a2936787c417a8f573813e3234743 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:31.724451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09188-a15b-4660-96f4-f364d58fc08e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:31.724968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09188-a15b-4660-96f4-f364d58fc08e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:31.725212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d09188-a15b-4660-96f4-f364d58fc08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:31.743217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a9d09188-a15b-4660-96f4-f364d58fc08e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:31.744615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a9d09188-a15b-4660-96f4-f364d58fc08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:31.805611 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1283/2580] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:31.813425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161da7d-7127-42a6-bbf0-22f08270cf22 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:31.813926 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161da7d-7127-42a6-bbf0-22f08270cf22 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:31.814026 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6161da7d-7127-42a6-bbf0-22f08270cf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:31.833722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6161da7d-7127-42a6-bbf0-22f08270cf22 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:31.835518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6161da7d-7127-42a6-bbf0-22f08270cf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:31.843446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1298/2581] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:31.847631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d09188-a15b-4660-96f4-f364d58fc08e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:31.857879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d09188-a15b-4660-96f4-f364d58fc08e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:31.862662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:31.863313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:31.863610 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:31.863887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:31.864646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:31.864941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:31.866634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:31.873597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:31.878857 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1284/2582] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:31.883792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161da7d-7127-42a6-bbf0-22f08270cf22 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:31.893192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6161da7d-7127-42a6-bbf0-22f08270cf22 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:31.898507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c11d5-5aca-4c6e-b7de-37f9732fee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:31.899201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c11d5-5aca-4c6e-b7de-37f9732fee7b admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:31.906307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c11d5-5aca-4c6e-b7de-37f9732fee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:31.906307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c11d5-5aca-4c6e-b7de-37f9732fee7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:31.906307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c11d5-5aca-4c6e-b7de-37f9732fee7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:31.906307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c11d5-5aca-4c6e-b7de-37f9732fee7b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:31.989247 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1299/2583] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:31 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:31.993917 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:32.004077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0fb41-7f6d-46b7-97fd-b4ff39c133c8 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:32.059987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756ae30-db7b-4806-8394-bba0e4cab5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:32.060695 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756ae30-db7b-4806-8394-bba0e4cab5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:32.060926 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756ae30-db7b-4806-8394-bba0e4cab5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:32.061233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756ae30-db7b-4806-8394-bba0e4cab5e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:32.061909 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756ae30-db7b-4806-8394-bba0e4cab5e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:32.063816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756ae30-db7b-4806-8394-bba0e4cab5e1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:32.182748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1285/2584] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:31 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:32.186558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c11d5-5aca-4c6e-b7de-37f9732fee7b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:32.197264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c11d5-5aca-4c6e-b7de-37f9732fee7b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:32.201897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47057804-b16a-4800-872f-2865509ba68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:32.202545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057804-b16a-4800-872f-2865509ba68d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:32.202830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057804-b16a-4800-872f-2865509ba68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:32.203105 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47057804-b16a-4800-872f-2865509ba68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:32.203824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47057804-b16a-4800-872f-2865509ba68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:32.205446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47057804-b16a-4800-872f-2865509ba68d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:32.217144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1300/2585] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:32.225603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756ae30-db7b-4806-8394-bba0e4cab5e1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:32.235413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756ae30-db7b-4806-8394-bba0e4cab5e1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:32.240071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c94ac-e170-4fac-a5ee-3b394493d6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:32.241255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c94ac-e170-4fac-a5ee-3b394493d6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:32.241544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c94ac-e170-4fac-a5ee-3b394493d6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:32.241820 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c94ac-e170-4fac-a5ee-3b394493d6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:32.242539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c94ac-e170-4fac-a5ee-3b394493d6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:32.244132 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c94ac-e170-4fac-a5ee-3b394493d6a6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:32.254163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1286/2586] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:32.258782 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47057804-b16a-4800-872f-2865509ba68d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:32.268610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47057804-b16a-4800-872f-2865509ba68d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:32.275536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:32.275536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:32.275536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:32.275536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] PATH_INFO: `/v3/projects/fe26d3e7624e4590b14313a6123daf9d/users/c678f9db7d884a24ac6d33ea2d6363ec/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:32.290771 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe26d3e7624e4590b14313a6123daf9d, user_id=c678f9db7d884a24ac6d33ea2d6363ec, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:32.293553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:32.370891 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1301/2587] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:32 2022] PUT /identity/v3/projects/fe26d3e7624e4590b14313a6123daf9d/users/c678f9db7d884a24ac6d33ea2d6363ec/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:32.382574 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d12f75-1e4e-4b2f-be42-274da0bdf2f0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:32.382876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d12f75-1e4e-4b2f-be42-274da0bdf2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:32.383159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d12f75-1e4e-4b2f-be42-274da0bdf2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:32.409794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-90d12f75-1e4e-4b2f-be42-274da0bdf2f0 None None] MFA Rules not processed for user `c678f9db7d884a24ac6d33ea2d6363ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:32.411034 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-90d12f75-1e4e-4b2f-be42-274da0bdf2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:32.474709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1287/2588] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:32.485987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:32.487652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:32.549083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8cb8e-6c4a-4eb9-8747-828c3cb4729b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:32.553644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfdc58-9856-4b67-9bf6-17796716ae7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:32.554296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfdc58-9856-4b67-9bf6-17796716ae7f service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:32.554577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfdc58-9856-4b67-9bf6-17796716ae7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:32.554850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfdc58-9856-4b67-9bf6-17796716ae7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:32.563676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfdc58-9856-4b67-9bf6-17796716ae7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:32.565189 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfdc58-9856-4b67-9bf6-17796716ae7f service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:32.574878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1302/2589] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:33.565163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d12f75-1e4e-4b2f-be42-274da0bdf2f0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:33.566675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-90d12f75-1e4e-4b2f-be42-274da0bdf2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:33.635152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d12f75-1e4e-4b2f-be42-274da0bdf2f0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:33.639948 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:33.640852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:33.641191 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:33.641470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] PATH_INFO: `/v3/users/760513191e2d4b38b3dfb1bb7c1a89f5` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:33.642805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=760513191e2d4b38b3dfb1bb7c1a89f5)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:33.644828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:33.651301 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:33.905501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760513191e2d4b38b3dfb1bb7c1a89f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:33.919019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760513191e2d4b38b3dfb1bb7c1a89f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:33.920235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760513191e2d4b38b3dfb1bb7c1a89f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:33.920353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760513191e2d4b38b3dfb1bb7c1a89f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:33.987456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760513191e2d4b38b3dfb1bb7c1a89f5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:34.001698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1288/2590] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:33 2022] DELETE /identity/v3/users/760513191e2d4b38b3dfb1bb7c1a89f5 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:34.006245 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8509-6e55-488c-b3dc-8ec26d6d69d0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:34.006725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8509-6e55-488c-b3dc-8ec26d6d69d0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:34.006824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8509-6e55-488c-b3dc-8ec26d6d69d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:34.023535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3aad8509-6e55-488c-b3dc-8ec26d6d69d0 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:34.024719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3aad8509-6e55-488c-b3dc-8ec26d6d69d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:34.082374 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1303/2591] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:34.092319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:34.093765 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:34.151075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5777b6-c27b-4ed5-88ce-27ab20a60420 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:34.154701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:34.155340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:34.155626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:34.155906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:34.165637 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:34.167071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:34.176468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1289/2592] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:34.332840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad8509-6e55-488c-b3dc-8ec26d6d69d0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:34.343304 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad8509-6e55-488c-b3dc-8ec26d6d69d0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:34.347347 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:34.347987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:34.348267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:34.348538 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:34.351099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:34.363594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:34.365081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:34.375147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1304/2593] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:34.607449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:34.609960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:34.633985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c77c1d-65cc-4c6b-8436-dccc3e53ca45 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:34.649737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:34.650615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:34.650886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:34.651188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] PATH_INFO: `/v3/projects/c7f2e71c1ea8422b9f13d34056f29a9d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:34.652571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7f2e71c1ea8422b9f13d34056f29a9d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:34.654533 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:34.762589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7f2e71c1ea8422b9f13d34056f29a9d', 'request_id': 'req-0737f85a-9a61-44f4-8193-acb26a2636ee'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:34.770080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7f2e71c1ea8422b9f13d34056f29a9d', 'request_id': 'req-0737f85a-9a61-44f4-8193-acb26a2636ee'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:34.776789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7f2e71c1ea8422b9f13d34056f29a9d', 'request_id': 'req-0737f85a-9a61-44f4-8193-acb26a2636ee'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:34.838853 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] The token cache is being invalidate because project c7f2e71c1ea8422b9f13d34056f29a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:34.839137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:34.839849 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:34.841682 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1290/2594] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:34 2022] DELETE /identity/v3/projects/c7f2e71c1ea8422b9f13d34056f29a9d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:34.845518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:34.856321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e61b-924f-4d39-9704-021532e930f6 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:34.860446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:34.927483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:34.928478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:34.928787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:34.929067 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] PATH_INFO: `/v3/users/2c2440089c534450b24845482a7fb0bb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:34.930322 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2440089c534450b24845482a7fb0bb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:34.932270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:35.150334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2440089c534450b24845482a7fb0bb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:35.157605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2440089c534450b24845482a7fb0bb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:35.169793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2440089c534450b24845482a7fb0bb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:35.170506 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2440089c534450b24845482a7fb0bb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:35.267215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e80d4210-3916-452b-9d6e-270bfa864a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2440089c534450b24845482a7fb0bb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:35.274492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1305/2595] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:34 2022] DELETE /identity/v3/users/2c2440089c534450b24845482a7fb0bb => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:35.976896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:35.976896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:35.989881 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737f85a-9a61-44f4-8193-acb26a2636ee admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:36.000105 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:36.055836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:36.056722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:36.057007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:36.057319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] PATH_INFO: `/v3/projects/dc3eae34c82744f6a4221a942d434d6f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:36.058582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3eae34c82744f6a4221a942d434d6f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:36.060624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:36.219794 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3eae34c82744f6a4221a942d434d6f', 'request_id': 'req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:36.227268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3eae34c82744f6a4221a942d434d6f', 'request_id': 'req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:36.233991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3eae34c82744f6a4221a942d434d6f', 'request_id': 'req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:36.348863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] The token cache is being invalidate because project dc3eae34c82744f6a4221a942d434d6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:36.349276 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:36.350056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-d6f6ae82-1cf1-4891-89f1-7c7be49d9fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:36.352353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1291/2596] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:35 2022] DELETE /identity/v3/projects/dc3eae34c82744f6a4221a942d434d6f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:36.361723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cded90-28b3-42da-82f9-5b55a2b87174 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:36.361723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cded90-28b3-42da-82f9-5b55a2b87174 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:36.361723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cded90-28b3-42da-82f9-5b55a2b87174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:36.382830 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f4cded90-28b3-42da-82f9-5b55a2b87174 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:36.384285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f4cded90-28b3-42da-82f9-5b55a2b87174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:36.449262 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1306/2597] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:36.457758 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1f767-49c0-4f51-8a15-e5748ca4b846 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:36.458264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1f767-49c0-4f51-8a15-e5748ca4b846 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:36.458622 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1f767-49c0-4f51-8a15-e5748ca4b846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:36.480392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-6cd1f767-49c0-4f51-8a15-e5748ca4b846 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:36.481789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6cd1f767-49c0-4f51-8a15-e5748ca4b846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:36.493184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1292/2598] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:36.494536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cded90-28b3-42da-82f9-5b55a2b87174 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:36.504765 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cded90-28b3-42da-82f9-5b55a2b87174 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:36.511045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:36.511045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:36.511045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:36.511045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:36.511446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:36.511685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:36.513306 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:36.529484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1307/2599] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:36.530802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1f767-49c0-4f51-8a15-e5748ca4b846 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:36.541462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1f767-49c0-4f51-8a15-e5748ca4b846 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:36.546513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa778d-dac7-472a-a90f-9da7ed16bdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:36.547285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa778d-dac7-472a-a90f-9da7ed16bdf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:36.547674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa778d-dac7-472a-a90f-9da7ed16bdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:36.548054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa778d-dac7-472a-a90f-9da7ed16bdf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:36.548847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa778d-dac7-472a-a90f-9da7ed16bdf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:36.550904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa778d-dac7-472a-a90f-9da7ed16bdf1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:36.706633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1293/2600] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:36 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:36.708156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:36.720799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92dc3d-e20c-4b73-ad7f-69d3d03263b4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:36.735651 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:36.806899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff10bc1-ddc5-4d75-9549-f9d92aa72841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:36.807584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff10bc1-ddc5-4d75-9549-f9d92aa72841 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:36.807873 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff10bc1-ddc5-4d75-9549-f9d92aa72841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:36.808157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff10bc1-ddc5-4d75-9549-f9d92aa72841 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:36.808859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff10bc1-ddc5-4d75-9549-f9d92aa72841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:36.810849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff10bc1-ddc5-4d75-9549-f9d92aa72841 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:36.960403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1308/2601] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:36 2022] POST /identity/v3/users => generated 286 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:36.964493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa778d-dac7-472a-a90f-9da7ed16bdf1 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:36.975666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa778d-dac7-472a-a90f-9da7ed16bdf1 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:36.984142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e4b87-5ce4-45ca-acff-4d7f1ceeb747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:36.984921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4b87-5ce4-45ca-acff-4d7f1ceeb747 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:36.985445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4b87-5ce4-45ca-acff-4d7f1ceeb747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:36.985903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e4b87-5ce4-45ca-acff-4d7f1ceeb747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:36.986826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e4b87-5ce4-45ca-acff-4d7f1ceeb747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:36.993932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e4b87-5ce4-45ca-acff-4d7f1ceeb747 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:37.008335 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1294/2602] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:37.012138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff10bc1-ddc5-4d75-9549-f9d92aa72841 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:37.028447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff10bc1-ddc5-4d75-9549-f9d92aa72841 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:37.034469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e725b-3450-4ae3-ba62-419fcb35bca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:37.035505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e725b-3450-4ae3-ba62-419fcb35bca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.035806 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e725b-3450-4ae3-ba62-419fcb35bca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.036636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e725b-3450-4ae3-ba62-419fcb35bca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.037676 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e725b-3450-4ae3-ba62-419fcb35bca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:37.039586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e725b-3450-4ae3-ba62-419fcb35bca5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:37.051775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1309/2603] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:37.056127 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e4b87-5ce4-45ca-acff-4d7f1ceeb747 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:37.066505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e4b87-5ce4-45ca-acff-4d7f1ceeb747 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:37.072090 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf87531-ebca-4058-b63d-7bb39fd03a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:37.078076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf87531-ebca-4058-b63d-7bb39fd03a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.078076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf87531-ebca-4058-b63d-7bb39fd03a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.078076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf87531-ebca-4058-b63d-7bb39fd03a1e admin admin] PATH_INFO: `/v3/projects/f203934f796346bb954177203dffe764/users/8efa009d8c9341d6b783a5aea6a218f2/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.090777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf87531-ebca-4058-b63d-7bb39fd03a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f203934f796346bb954177203dffe764, user_id=8efa009d8c9341d6b783a5aea6a218f2, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:37.093737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf87531-ebca-4058-b63d-7bb39fd03a1e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:37.191485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1295/2604] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:37 2022] PUT /identity/v3/projects/f203934f796346bb954177203dffe764/users/8efa009d8c9341d6b783a5aea6a218f2/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:37.202539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9c775-8f41-4162-aaaa-51a501875571 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.203146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9c775-8f41-4162-aaaa-51a501875571 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.203445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9c775-8f41-4162-aaaa-51a501875571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.229876 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-02f9c775-8f41-4162-aaaa-51a501875571 None None] MFA Rules not processed for user `8efa009d8c9341d6b783a5aea6a218f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:37.231188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-02f9c775-8f41-4162-aaaa-51a501875571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:37.294815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1310/2605] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:37.299402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf87531-ebca-4058-b63d-7bb39fd03a1e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:37.347235 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:37.362976 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf87531-ebca-4058-b63d-7bb39fd03a1e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:37.372283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8620-137a-4bdf-a944-e6e8af3f7aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:37.372283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8620-137a-4bdf-a944-e6e8af3f7aee admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.372283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8620-137a-4bdf-a944-e6e8af3f7aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.372283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a8620-137a-4bdf-a944-e6e8af3f7aee admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.372947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8620-137a-4bdf-a944-e6e8af3f7aee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:37.372947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a8620-137a-4bdf-a944-e6e8af3f7aee admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:37.451098 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1296/2606] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:37 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:37.455235 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9c775-8f41-4162-aaaa-51a501875571 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:37.467118 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9c775-8f41-4162-aaaa-51a501875571 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:37.541829 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a306cb-87ac-466d-aca9-5784fc4c13fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:37.542543 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a306cb-87ac-466d-aca9-5784fc4c13fa admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.542810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a306cb-87ac-466d-aca9-5784fc4c13fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.543082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a306cb-87ac-466d-aca9-5784fc4c13fa admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.544239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a306cb-87ac-466d-aca9-5784fc4c13fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:37.546925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a306cb-87ac-466d-aca9-5784fc4c13fa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:37.652244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1311/2607] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:37 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:37.659952 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8620-137a-4bdf-a944-e6e8af3f7aee admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:37.667631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a8620-137a-4bdf-a944-e6e8af3f7aee admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:37.674371 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:37.675203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.675543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.675891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.676699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:37.679190 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:37.690374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1297/2608] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:37.694814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a306cb-87ac-466d-aca9-5784fc4c13fa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:37.704695 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a306cb-87ac-466d-aca9-5784fc4c13fa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:37.709602 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad89d7-c290-4032-a9eb-da8b7c1c2113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:37.710478 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad89d7-c290-4032-a9eb-da8b7c1c2113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.710802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad89d7-c290-4032-a9eb-da8b7c1c2113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.711217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad89d7-c290-4032-a9eb-da8b7c1c2113 admin admin] PATH_INFO: `/v3/projects/946e698d460442d9a9c4d663b99d556d/users/865d3e1b28f244d09b8d74411ed5c5cb/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.742452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad89d7-c290-4032-a9eb-da8b7c1c2113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946e698d460442d9a9c4d663b99d556d, user_id=865d3e1b28f244d09b8d74411ed5c5cb, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:37.747962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad89d7-c290-4032-a9eb-da8b7c1c2113 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:37.754767 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:37.756675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:37.840147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:37.841719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1312/2609] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:37 2022] PUT /identity/v3/projects/946e698d460442d9a9c4d663b99d556d/users/865d3e1b28f244d09b8d74411ed5c5cb/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:37.842657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad89d7-c290-4032-a9eb-da8b7c1c2113 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:37.860890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ebc7e-8003-4dc4-98f0-d9a716d1ac05 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:37.944353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:37.945356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.945840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.947391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.953793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:37.992633 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad89d7-c290-4032-a9eb-da8b7c1c2113 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:37.996997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a7a7b-179a-4711-ac0e-92dd6c7062a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:37.997717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a7a7b-179a-4711-ac0e-92dd6c7062a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:37.998007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a7a7b-179a-4711-ac0e-92dd6c7062a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:37.998297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a7a7b-179a-4711-ac0e-92dd6c7062a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:37.999035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a7a7b-179a-4711-ac0e-92dd6c7062a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:38.000987 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a7a7b-179a-4711-ac0e-92dd6c7062a9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:38.015860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1313/2610] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:38.019988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a7a7b-179a-4711-ac0e-92dd6c7062a9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:38.045159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a7a7b-179a-4711-ac0e-92dd6c7062a9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:38.053271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586178fe-3b2b-4d03-a776-e1729660494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:38.054532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586178fe-3b2b-4d03-a776-e1729660494a admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:38.054883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586178fe-3b2b-4d03-a776-e1729660494a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:38.055295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586178fe-3b2b-4d03-a776-e1729660494a admin admin] PATH_INFO: `/v3/projects/946e698d460442d9a9c4d663b99d556d/users/865d3e1b28f244d09b8d74411ed5c5cb/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:38.062294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586178fe-3b2b-4d03-a776-e1729660494a admin admin] RBAC: Authorizing `identity:create_grant(project_id=946e698d460442d9a9c4d663b99d556d, user_id=865d3e1b28f244d09b8d74411ed5c5cb, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:38.066420 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586178fe-3b2b-4d03-a776-e1729660494a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:38.081938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:38.084873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:38.085093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1314/2611] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:38 2022] PUT /identity/v3/projects/946e698d460442d9a9c4d663b99d556d/users/865d3e1b28f244d09b8d74411ed5c5cb/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:38.095237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0b86-71e9-4626-bcc7-2ac60fb273b4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:38.095571 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0b86-71e9-4626-bcc7-2ac60fb273b4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:38.095875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a0b86-71e9-4626-bcc7-2ac60fb273b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:38.105993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1298/2612] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:38.129549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7a1a0b86-71e9-4626-bcc7-2ac60fb273b4 None None] MFA Rules not processed for user `865d3e1b28f244d09b8d74411ed5c5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:38.131542 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7a1a0b86-71e9-4626-bcc7-2ac60fb273b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:38.265869 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1315/2613] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:38.279013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:38.280426 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:38.302683 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:38.358707 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b688fdb-d426-4506-b4a0-e5a7017e09b6 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:38.461281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80c526-26e0-4e1f-95dc-41cad60a4216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:38.462268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80c526-26e0-4e1f-95dc-41cad60a4216 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:38.462565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80c526-26e0-4e1f-95dc-41cad60a4216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:38.462850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80c526-26e0-4e1f-95dc-41cad60a4216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:38.479549 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80c526-26e0-4e1f-95dc-41cad60a4216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:38.481630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80c526-26e0-4e1f-95dc-41cad60a4216 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:38.498342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1299/2614] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:38.772525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a0b86-71e9-4626-bcc7-2ac60fb273b4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:38.774672 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7a1a0b86-71e9-4626-bcc7-2ac60fb273b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:38.853545 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:38.860766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a0b86-71e9-4626-bcc7-2ac60fb273b4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:38.866210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:38.867147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:38.868010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:38.868010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] PATH_INFO: `/v3/users/e0eb0ab5585747d098bbb0b18770f4c5` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:38.869095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0eb0ab5585747d098bbb0b18770f4c5)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:38.871451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:39.113238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0eb0ab5585747d098bbb0b18770f4c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:39.141669 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0eb0ab5585747d098bbb0b18770f4c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:39.167849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0eb0ab5585747d098bbb0b18770f4c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:39.168730 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0eb0ab5585747d098bbb0b18770f4c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:39.254457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0eb0ab5585747d098bbb0b18770f4c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:39.262360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1316/2615] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:38 2022] DELETE /identity/v3/users/e0eb0ab5585747d098bbb0b18770f4c5 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:39.555821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80c526-26e0-4e1f-95dc-41cad60a4216 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:39.580838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80c526-26e0-4e1f-95dc-41cad60a4216 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:39.608074 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:39.613747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:39.666457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:39.667347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:39.667639 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:39.667924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] PATH_INFO: `/v3/projects/3611c72fb0b8472da98ba9b0c0d86843` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:39.669187 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3611c72fb0b8472da98ba9b0c0d86843)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:39.671478 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:39.718969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04447e-3640-40cd-8be6-7558c5d18a34 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:39.723677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:39.724546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:39.724898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:39.725243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] PATH_INFO: `/v3/users/8efa009d8c9341d6b783a5aea6a218f2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:39.727774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8efa009d8c9341d6b783a5aea6a218f2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:39.729811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:39.882427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3611c72fb0b8472da98ba9b0c0d86843', 'request_id': 'req-1853e4eb-144b-4cc7-b280-aa4f7cb84424'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:39.889850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3611c72fb0b8472da98ba9b0c0d86843', 'request_id': 'req-1853e4eb-144b-4cc7-b280-aa4f7cb84424'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:39.898219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3611c72fb0b8472da98ba9b0c0d86843', 'request_id': 'req-1853e4eb-144b-4cc7-b280-aa4f7cb84424'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:40.049779 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] The token cache is being invalidate because project 3611c72fb0b8472da98ba9b0c0d86843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:40.050137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:40.050375 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8efa009d8c9341d6b783a5aea6a218f2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:40.050884 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1853e4eb-144b-4cc7-b280-aa4f7cb84424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:40.059221 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8efa009d8c9341d6b783a5aea6a218f2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:40.062704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1300/2616] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:39 2022] DELETE /identity/v3/projects/3611c72fb0b8472da98ba9b0c0d86843 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:40.069229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64fb4f-bdee-49bb-8bc8-59d44e3741d2 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.069804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64fb4f-bdee-49bb-8bc8-59d44e3741d2 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.070480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64fb4f-bdee-49bb-8bc8-59d44e3741d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.078555 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8efa009d8c9341d6b783a5aea6a218f2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:40.079327 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8efa009d8c9341d6b783a5aea6a218f2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:40.083639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:40.092005 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:40.096954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-ba64fb4f-bdee-49bb-8bc8-59d44e3741d2 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:40.096954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ba64fb4f-bdee-49bb-8bc8-59d44e3741d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:40.153046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a1d0f2b9-e5eb-4bfc-86fb-6b5d27041555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8efa009d8c9341d6b783a5aea6a218f2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:40.163179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1317/2617] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:39 2022] DELETE /identity/v3/users/8efa009d8c9341d6b783a5aea6a218f2 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:40.167558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d4df-cf2e-4047-9640-ea6f77973a02 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.167908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d4df-cf2e-4047-9640-ea6f77973a02 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.168295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08d4df-cf2e-4047-9640-ea6f77973a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.184079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1301/2618] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:40.192369 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07f099-288c-48f7-9a60-7eb17910e58a None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.193134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07f099-288c-48f7-9a60-7eb17910e58a None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.193134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07f099-288c-48f7-9a60-7eb17910e58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.201064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7d08d4df-cf2e-4047-9640-ea6f77973a02 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:40.202780 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7d08d4df-cf2e-4047-9640-ea6f77973a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:40.219796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dd07f099-288c-48f7-9a60-7eb17910e58a None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:40.221714 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dd07f099-288c-48f7-9a60-7eb17910e58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:40.296438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1318/2619] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:40.306332 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08d4df-cf2e-4047-9640-ea6f77973a02 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:40.307951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7d08d4df-cf2e-4047-9640-ea6f77973a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:40.309424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1302/2620] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:40.313189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07f099-288c-48f7-9a60-7eb17910e58a None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:40.325617 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07f099-288c-48f7-9a60-7eb17910e58a None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:40.331048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:40.331865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.332250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.332632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.333536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:40.334008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:40.336445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:40.361554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1303/2621] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:40.364413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:40.385208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2549384-1e43-4e39-a511-aebcf5218f3a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:40.390966 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb9d17-f331-4064-a0af-169756af5a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:40.391947 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9d17-f331-4064-a0af-169756af5a67 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.392341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9d17-f331-4064-a0af-169756af5a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.392723 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9d17-f331-4064-a0af-169756af5a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.393553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb9d17-f331-4064-a0af-169756af5a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:40.395596 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb9d17-f331-4064-a0af-169756af5a67 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:40.408591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08d4df-cf2e-4047-9640-ea6f77973a02 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:40.414248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:40.415099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.415557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.415984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.426993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:40.430185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:40.443349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1319/2622] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:40.545509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:40.546977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:40.565342 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1304/2623] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:40 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:40.571633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb9d17-f331-4064-a0af-169756af5a67 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:40.682724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b48065-86b9-45e8-8a09-011df667b6c0 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:40.691891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb9d17-f331-4064-a0af-169756af5a67 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:40.707608 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42b5a8-dcda-476c-a2b9-dfd783b07757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:40.708542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b5a8-dcda-476c-a2b9-dfd783b07757 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.708897 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b5a8-dcda-476c-a2b9-dfd783b07757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.711230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42b5a8-dcda-476c-a2b9-dfd783b07757 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.712214 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42b5a8-dcda-476c-a2b9-dfd783b07757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:40.718681 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf42b5a8-dcda-476c-a2b9-dfd783b07757 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:40.723784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:40.766584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:40.767410 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.767758 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.768041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.770513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:40.799901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:40.800017 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:40.817158 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1320/2624] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:40.834194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1305/2625] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:40 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:40.839954 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:40.858691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40715ae8-76ed-4805-9c6a-f0ae905f7d4f service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:40.863626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280ee93-1968-4075-abec-afb315a98a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:40.864314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280ee93-1968-4075-abec-afb315a98a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.864603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280ee93-1968-4075-abec-afb315a98a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.864890 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280ee93-1968-4075-abec-afb315a98a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.866761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280ee93-1968-4075-abec-afb315a98a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:40.867849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280ee93-1968-4075-abec-afb315a98a1c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:40.883920 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1321/2626] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:40.884911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42b5a8-dcda-476c-a2b9-dfd783b07757 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:40.891982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:40.913519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42b5a8-dcda-476c-a2b9-dfd783b07757 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:40.934977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:40.937675 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.938170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.938748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:40.939685 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:40.943137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:40.955710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1306/2627] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:40.960670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280ee93-1968-4075-abec-afb315a98a1c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:40.976116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280ee93-1968-4075-abec-afb315a98a1c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:40.984233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ff26e-dffb-4b04-92e5-2a6bb7f7cf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:40.985973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ff26e-dffb-4b04-92e5-2a6bb7f7cf03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:40.986455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ff26e-dffb-4b04-92e5-2a6bb7f7cf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:40.986895 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ff26e-dffb-4b04-92e5-2a6bb7f7cf03 admin admin] PATH_INFO: `/v3/projects/0bd351ab95cf4b3d92dd7de1135d7b4a/users/581bf68d6942466b9143b5535e35e580/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:41.023080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ff26e-dffb-4b04-92e5-2a6bb7f7cf03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd351ab95cf4b3d92dd7de1135d7b4a, user_id=581bf68d6942466b9143b5535e35e580, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:41.025786 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ff26e-dffb-4b04-92e5-2a6bb7f7cf03 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:41.135987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:41.143884 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1322/2628] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:40 2022] PUT /identity/v3/projects/0bd351ab95cf4b3d92dd7de1135d7b4a/users/581bf68d6942466b9143b5535e35e580/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:41.144319 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:41.153819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233a24a-d3d9-4ff9-b640-a13367d6db36 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:41.154238 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233a24a-d3d9-4ff9-b640-a13367d6db36 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:41.154620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233a24a-d3d9-4ff9-b640-a13367d6db36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:41.200993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559104db-f424-4db7-a03b-d5c197644027 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:41.211674 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5233a24a-d3d9-4ff9-b640-a13367d6db36 None None] MFA Rules not processed for user `581bf68d6942466b9143b5535e35e580`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:41.213831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5233a24a-d3d9-4ff9-b640-a13367d6db36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:41.301225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1323/2629] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:41.314006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233a24a-d3d9-4ff9-b640-a13367d6db36 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:41.316753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5233a24a-d3d9-4ff9-b640-a13367d6db36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:41.325555 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:41.326805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:41.326805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:41.327027 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] PATH_INFO: `/v3/projects/f203934f796346bb954177203dffe764` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:41.328261 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f203934f796346bb954177203dffe764)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:41.330565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:41.410106 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233a24a-d3d9-4ff9-b640-a13367d6db36 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:41.415945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe0a0d-1af0-494d-a6e5-fd55c1b86189 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:41.416640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe0a0d-1af0-494d-a6e5-fd55c1b86189 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:41.417199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe0a0d-1af0-494d-a6e5-fd55c1b86189 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:41.417483 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe0a0d-1af0-494d-a6e5-fd55c1b86189 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:41.426673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe0a0d-1af0-494d-a6e5-fd55c1b86189 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:41.428404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe0a0d-1af0-494d-a6e5-fd55c1b86189 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:41.438053 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1324/2630] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:41.472730 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:41.476829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f203934f796346bb954177203dffe764', 'request_id': 'req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:41.488203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f203934f796346bb954177203dffe764', 'request_id': 'req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:41.496474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f203934f796346bb954177203dffe764', 'request_id': 'req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:41.600553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] The token cache is being invalidate because project f203934f796346bb954177203dffe764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:41.601752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:41.602525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:41.607317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1307/2631] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:41 2022] DELETE /identity/v3/projects/f203934f796346bb954177203dffe764 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:41.608754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe0a0d-1af0-494d-a6e5-fd55c1b86189 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:41.623712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe0a0d-1af0-494d-a6e5-fd55c1b86189 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:41.714641 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:41.716690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:41.716690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:41.716690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] PATH_INFO: `/v3/users/865d3e1b28f244d09b8d74411ed5c5cb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:41.717784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=865d3e1b28f244d09b8d74411ed5c5cb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:41.720361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:41.938849 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '865d3e1b28f244d09b8d74411ed5c5cb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:41.949462 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '865d3e1b28f244d09b8d74411ed5c5cb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:41.961962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '865d3e1b28f244d09b8d74411ed5c5cb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:41.963070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '865d3e1b28f244d09b8d74411ed5c5cb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:42.054687 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e53d189f-a5cb-4f92-9424-663e67058de9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '865d3e1b28f244d09b8d74411ed5c5cb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:42.062344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1325/2632] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:41 2022] DELETE /identity/v3/users/865d3e1b28f244d09b8d74411ed5c5cb => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:42.577777 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:42.577985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:42.591316 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91972bda-2a03-4f6d-a3bd-6f5a7cdaef4a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:42.655272 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:42.656154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:42.656459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:42.656726 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] PATH_INFO: `/v3/projects/946e698d460442d9a9c4d663b99d556d` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:42.657994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] RBAC: Authorizing `identity:delete_project(project_id=946e698d460442d9a9c4d663b99d556d)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:42.659954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:42.689782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:42.793460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946e698d460442d9a9c4d663b99d556d', 'request_id': 'req-b0472b1e-202d-4072-b371-d0f6c9a4366b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:42.799516 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946e698d460442d9a9c4d663b99d556d', 'request_id': 'req-b0472b1e-202d-4072-b371-d0f6c9a4366b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:42.807411 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946e698d460442d9a9c4d663b99d556d', 'request_id': 'req-b0472b1e-202d-4072-b371-d0f6c9a4366b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:42.970547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] The token cache is being invalidate because project 946e698d460442d9a9c4d663b99d556d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:42.970547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:42.971229 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b0472b1e-202d-4072-b371-d0f6c9a4366b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:42.972344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1308/2633] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:42 2022] DELETE /identity/v3/projects/946e698d460442d9a9c4d663b99d556d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:42.978197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d8d0-0967-44a8-8bf2-d591f61c39ee None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:42.979058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d8d0-0967-44a8-8bf2-d591f61c39ee None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:42.979364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58d8d0-0967-44a8-8bf2-d591f61c39ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.001195 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ae58d8d0-0967-44a8-8bf2-d591f61c39ee None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:43.002495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ae58d8d0-0967-44a8-8bf2-d591f61c39ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:43.066498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1326/2634] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:43.074664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1022330-67c3-46d2-b579-4752f1727580 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.075062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1022330-67c3-46d2-b579-4752f1727580 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.075349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1022330-67c3-46d2-b579-4752f1727580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.096851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d1022330-67c3-46d2-b579-4752f1727580 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:43.098144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d1022330-67c3-46d2-b579-4752f1727580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:43.106824 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1309/2635] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:43.111038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58d8d0-0967-44a8-8bf2-d591f61c39ee None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:43.121625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58d8d0-0967-44a8-8bf2-d591f61c39ee None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:43.126286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:43.126938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.127230 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.127495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.128260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:43.128625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:43.130229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:43.142281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1327/2636] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:43.147154 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1022330-67c3-46d2-b579-4752f1727580 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:43.156260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1022330-67c3-46d2-b579-4752f1727580 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:43.161217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f48d-010a-4c95-b43d-9626aab00b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:43.163197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f48d-010a-4c95-b43d-9626aab00b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.163197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f48d-010a-4c95-b43d-9626aab00b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.163197 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f48d-010a-4c95-b43d-9626aab00b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.163769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4f48d-010a-4c95-b43d-9626aab00b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:43.165713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4f48d-010a-4c95-b43d-9626aab00b5b admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:43.248985 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1310/2637] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:43 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:43.256434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:43.265845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ddd2f-bbd6-43c1-b897-1852b9b624f0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:43.275136 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:43.337192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a34c6b-9230-41e9-b3f8-40c3c93867ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:43.337868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a34c6b-9230-41e9-b3f8-40c3c93867ab admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.338162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a34c6b-9230-41e9-b3f8-40c3c93867ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.338445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a34c6b-9230-41e9-b3f8-40c3c93867ab admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.339157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a34c6b-9230-41e9-b3f8-40c3c93867ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:43.341087 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a34c6b-9230-41e9-b3f8-40c3c93867ab admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:43.483034 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1328/2638] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:43 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:43.484263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f48d-010a-4c95-b43d-9626aab00b5b admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:43.494615 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f48d-010a-4c95-b43d-9626aab00b5b admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:43.500245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56d934-0fef-4cc4-be5f-a51f575e43e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:43.500379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d934-0fef-4cc4-be5f-a51f575e43e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.500673 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d934-0fef-4cc4-be5f-a51f575e43e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.500958 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d934-0fef-4cc4-be5f-a51f575e43e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.501718 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56d934-0fef-4cc4-be5f-a51f575e43e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:43.503330 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56d934-0fef-4cc4-be5f-a51f575e43e2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:43.514324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1311/2639] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:43.518368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a34c6b-9230-41e9-b3f8-40c3c93867ab admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:43.529083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a34c6b-9230-41e9-b3f8-40c3c93867ab admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:43.533911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46195416-3804-47b8-8978-9e6ef21aa66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:43.534645 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46195416-3804-47b8-8978-9e6ef21aa66c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.534945 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46195416-3804-47b8-8978-9e6ef21aa66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.535226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46195416-3804-47b8-8978-9e6ef21aa66c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.535978 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46195416-3804-47b8-8978-9e6ef21aa66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:43.537640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46195416-3804-47b8-8978-9e6ef21aa66c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:43.547816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1329/2640] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:43.551865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56d934-0fef-4cc4-be5f-a51f575e43e2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:43.561903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56d934-0fef-4cc4-be5f-a51f575e43e2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:43.569201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f88c3-2c3e-4c85-b436-3c439526b3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:43.569201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f88c3-2c3e-4c85-b436-3c439526b3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.569201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f88c3-2c3e-4c85-b436-3c439526b3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.569201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f88c3-2c3e-4c85-b436-3c439526b3a6 admin admin] PATH_INFO: `/v3/projects/f6c6cae5e8724497a9b5bf14688a8c5c/users/a3d5e236409a499f98107b450216514b/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.582880 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f88c3-2c3e-4c85-b436-3c439526b3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c6cae5e8724497a9b5bf14688a8c5c, user_id=a3d5e236409a499f98107b450216514b, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:43.586698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f88c3-2c3e-4c85-b436-3c439526b3a6 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:43.694438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1312/2641] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:43 2022] PUT /identity/v3/projects/f6c6cae5e8724497a9b5bf14688a8c5c/users/a3d5e236409a499f98107b450216514b/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:43.704845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a255aa-8724-4fff-904c-bd789f6afe01 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.705170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a255aa-8724-4fff-904c-bd789f6afe01 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.705458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a255aa-8724-4fff-904c-bd789f6afe01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.733348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d9a255aa-8724-4fff-904c-bd789f6afe01 None None] MFA Rules not processed for user `a3d5e236409a499f98107b450216514b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:43.734567 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d9a255aa-8724-4fff-904c-bd789f6afe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:43.802176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1330/2642] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:43.809356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f88c3-2c3e-4c85-b436-3c439526b3a6 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:43.898092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f88c3-2c3e-4c85-b436-3c439526b3a6 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:43.914215 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd86e6d-cbf3-4862-9b1d-3ad389fa4cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:43.915033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd86e6d-cbf3-4862-9b1d-3ad389fa4cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:43.915360 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd86e6d-cbf3-4862-9b1d-3ad389fa4cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:43.917445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd86e6d-cbf3-4862-9b1d-3ad389fa4cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:43.918526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd86e6d-cbf3-4862-9b1d-3ad389fa4cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:43.925238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd86e6d-cbf3-4862-9b1d-3ad389fa4cfc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:44.063638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1313/2643] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:43 2022] POST /identity/v3/projects => generated 381 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:44.067516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a255aa-8724-4fff-904c-bd789f6afe01 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:44.079360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a255aa-8724-4fff-904c-bd789f6afe01 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:44.138126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783146ec-0a49-47ec-8557-6c2124949455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:44.139042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783146ec-0a49-47ec-8557-6c2124949455 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:44.139329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783146ec-0a49-47ec-8557-6c2124949455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:44.139609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783146ec-0a49-47ec-8557-6c2124949455 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:44.140349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783146ec-0a49-47ec-8557-6c2124949455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:44.142383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783146ec-0a49-47ec-8557-6c2124949455 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:44.271778 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1331/2644] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:44 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:44.275809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd86e6d-cbf3-4862-9b1d-3ad389fa4cfc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:44.286184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd86e6d-cbf3-4862-9b1d-3ad389fa4cfc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:44.291467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f9681-dce8-4e57-9cd5-841d065d15cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:44.292155 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f9681-dce8-4e57-9cd5-841d065d15cc admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:44.292432 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f9681-dce8-4e57-9cd5-841d065d15cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:44.292782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f9681-dce8-4e57-9cd5-841d065d15cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:44.293567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f9681-dce8-4e57-9cd5-841d065d15cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:44.298964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f9681-dce8-4e57-9cd5-841d065d15cc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:44.307328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1314/2645] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:44.311566 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783146ec-0a49-47ec-8557-6c2124949455 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:44.318236 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:44.322317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783146ec-0a49-47ec-8557-6c2124949455 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:44.327054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9018d-979a-4bed-935a-b56e683f02de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:44.328141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9018d-979a-4bed-935a-b56e683f02de admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:44.328539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9018d-979a-4bed-935a-b56e683f02de admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:44.328833 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9018d-979a-4bed-935a-b56e683f02de admin admin] PATH_INFO: `/v3/projects/8d92bb261ea94ab9a8b806e426d512d7/users/b68c3025dc294fdaabd502236c446c93/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:44.345656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9018d-979a-4bed-935a-b56e683f02de admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d92bb261ea94ab9a8b806e426d512d7, user_id=b68c3025dc294fdaabd502236c446c93, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:44.348386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9018d-979a-4bed-935a-b56e683f02de admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:44.502282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1332/2646] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:44 2022] PUT /identity/v3/projects/8d92bb261ea94ab9a8b806e426d512d7/users/b68c3025dc294fdaabd502236c446c93/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:44.506281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f9681-dce8-4e57-9cd5-841d065d15cc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:44.516095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f9681-dce8-4e57-9cd5-841d065d15cc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:44.534153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:44.576410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18328588-2ad2-4e18-a01f-aed2859d2eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:44.581717 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18328588-2ad2-4e18-a01f-aed2859d2eea admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:44.582017 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18328588-2ad2-4e18-a01f-aed2859d2eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:44.584561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18328588-2ad2-4e18-a01f-aed2859d2eea admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:44.584561 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18328588-2ad2-4e18-a01f-aed2859d2eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:44.585283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18328588-2ad2-4e18-a01f-aed2859d2eea admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:44.604867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1315/2647] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:44.606636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9018d-979a-4bed-935a-b56e683f02de admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:44.620191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9018d-979a-4bed-935a-b56e683f02de admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:44.634175 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:44.635414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:44.635693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:44.635973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] PATH_INFO: `/v3/projects/8d92bb261ea94ab9a8b806e426d512d7/users/b68c3025dc294fdaabd502236c446c93/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:44.638639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d92bb261ea94ab9a8b806e426d512d7, user_id=b68c3025dc294fdaabd502236c446c93, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:44.641618 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:44.653885 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1333/2648] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:44 2022] PUT /identity/v3/projects/8d92bb261ea94ab9a8b806e426d512d7/users/b68c3025dc294fdaabd502236c446c93/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:44.667627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ef95a-e102-4efa-8cc7-ec9d45141f19 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:44.668344 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ef95a-e102-4efa-8cc7-ec9d45141f19 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:44.669259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ef95a-e102-4efa-8cc7-ec9d45141f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:44.707010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d30ef95a-e102-4efa-8cc7-ec9d45141f19 None None] MFA Rules not processed for user `b68c3025dc294fdaabd502236c446c93`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:44.708981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d30ef95a-e102-4efa-8cc7-ec9d45141f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:44.787776 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1316/2649] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:44.799578 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:44.801178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:44.860055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b9bd5-3a2d-4175-9717-8d59870e556e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:44.863774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:44.864513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:44.864871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:44.865250 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:44.875787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:44.877342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:44.888708 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1334/2650] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:44.914614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ef95a-e102-4efa-8cc7-ec9d45141f19 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:44.914614 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d30ef95a-e102-4efa-8cc7-ec9d45141f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:44.983663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ef95a-e102-4efa-8cc7-ec9d45141f19 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:44.988640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:44.989464 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:44.989766 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:44.990031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] PATH_INFO: `/v3/users/c678f9db7d884a24ac6d33ea2d6363ec` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:44.991804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c678f9db7d884a24ac6d33ea2d6363ec)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:44.994020 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:44.994733 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:44.996448 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:45.008176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced992f-3f7f-4c76-9a0b-4f56295e463d service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:45.012794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fba73-5d39-456f-b89a-9732ff3448f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:45.012794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fba73-5d39-456f-b89a-9732ff3448f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:45.013139 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fba73-5d39-456f-b89a-9732ff3448f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:45.013496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fba73-5d39-456f-b89a-9732ff3448f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:45.075666 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fba73-5d39-456f-b89a-9732ff3448f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:45.077183 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fba73-5d39-456f-b89a-9732ff3448f2 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:45.086096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1335/2651] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:45.220039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:45.221771 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c678f9db7d884a24ac6d33ea2d6363ec'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:45.233903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c678f9db7d884a24ac6d33ea2d6363ec'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:45.234569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c678f9db7d884a24ac6d33ea2d6363ec'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:45.235180 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c678f9db7d884a24ac6d33ea2d6363ec'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:45.341705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c678f9db7d884a24ac6d33ea2d6363ec'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:45.354840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1317/2652] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:44 2022] DELETE /identity/v3/users/c678f9db7d884a24ac6d33ea2d6363ec => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:45.359313 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa01585-bdfd-4e8b-a334-c7e1587958e9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:45.359718 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa01585-bdfd-4e8b-a334-c7e1587958e9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:45.360002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa01585-bdfd-4e8b-a334-c7e1587958e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:45.377153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7fa01585-bdfd-4e8b-a334-c7e1587958e9 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:45.378283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7fa01585-bdfd-4e8b-a334-c7e1587958e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:45.440021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1336/2653] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:45.450482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:45.452492 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:45.511944 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db421700-04f6-4196-9a77-79955f710d35 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:45.515621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:45.516259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:45.516576 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:45.516941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:45.527341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:45.528961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:45.538529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1318/2654] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:45.958243 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa01585-bdfd-4e8b-a334-c7e1587958e9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:45.960261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7fa01585-bdfd-4e8b-a334-c7e1587958e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:45.977531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa01585-bdfd-4e8b-a334-c7e1587958e9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:45.982182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:45.983069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:45.983346 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:45.983565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] PATH_INFO: `/v3/projects/fe26d3e7624e4590b14313a6123daf9d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:45.984752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe26d3e7624e4590b14313a6123daf9d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:45.986689 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:45.989214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:46.132383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe26d3e7624e4590b14313a6123daf9d', 'request_id': 'req-c4846559-655b-42f1-a841-fd928ede8f30'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:46.139908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe26d3e7624e4590b14313a6123daf9d', 'request_id': 'req-c4846559-655b-42f1-a841-fd928ede8f30'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:46.145481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe26d3e7624e4590b14313a6123daf9d', 'request_id': 'req-c4846559-655b-42f1-a841-fd928ede8f30'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:46.227580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] The token cache is being invalidate because project fe26d3e7624e4590b14313a6123daf9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:46.227970 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:46.228443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c4846559-655b-42f1-a841-fd928ede8f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:46.230023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1337/2655] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:45 2022] DELETE /identity/v3/projects/fe26d3e7624e4590b14313a6123daf9d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:49.322079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:49.323520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:49.336654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01362bf7-073e-419c-9360-a8bb8c66783d service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:49.396702 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:49.397623 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:49.397906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:49.398208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] PATH_INFO: `/v3/users/a3d5e236409a499f98107b450216514b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:49.399507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3d5e236409a499f98107b450216514b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:49.401544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:49.588845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3d5e236409a499f98107b450216514b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:49.600529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3d5e236409a499f98107b450216514b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:49.601233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3d5e236409a499f98107b450216514b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:49.602435 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3d5e236409a499f98107b450216514b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:49.604937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:49.724735 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3d5e236409a499f98107b450216514b'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:49.737904 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1319/2656] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:49 2022] DELETE /identity/v3/users/a3d5e236409a499f98107b450216514b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:49.742914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51adf3-82a4-4d43-84b1-8038f0223a1b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:49.743392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51adf3-82a4-4d43-84b1-8038f0223a1b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:49.743534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51adf3-82a4-4d43-84b1-8038f0223a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:49.761750 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8c51adf3-82a4-4d43-84b1-8038f0223a1b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:49.762957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8c51adf3-82a4-4d43-84b1-8038f0223a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:49.828682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1338/2657] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:49.840256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:49.841657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:49.880505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51adf3-82a4-4d43-84b1-8038f0223a1b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:49.881950 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8c51adf3-82a4-4d43-84b1-8038f0223a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:49.893727 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51adf3-82a4-4d43-84b1-8038f0223a1b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:49.898294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:49.899697 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:49.899996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:49.900251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] PATH_INFO: `/v3/users/581bf68d6942466b9143b5535e35e580` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:49.901603 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=581bf68d6942466b9143b5535e35e580)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:49.903029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f83f1ac-1b71-4fac-a7ed-04735af7e8c3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:49.903526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:49.913494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67dc1e-22b6-4e74-8530-c85f4186ad86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:49.914231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67dc1e-22b6-4e74-8530-c85f4186ad86 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:49.917328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67dc1e-22b6-4e74-8530-c85f4186ad86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:49.917328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67dc1e-22b6-4e74-8530-c85f4186ad86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:49.927957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67dc1e-22b6-4e74-8530-c85f4186ad86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:49.929513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67dc1e-22b6-4e74-8530-c85f4186ad86 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:49.942104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1320/2658] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:50.166470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581bf68d6942466b9143b5535e35e580'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.173407 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581bf68d6942466b9143b5535e35e580'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.186741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581bf68d6942466b9143b5535e35e580'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.187451 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581bf68d6942466b9143b5535e35e580'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.264935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581bf68d6942466b9143b5535e35e580'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.272460 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1339/2659] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:49 2022] DELETE /identity/v3/users/581bf68d6942466b9143b5535e35e580 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:50.276938 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030d709-64e7-4413-a21d-6f133a1fbe7d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:50.279643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030d709-64e7-4413-a21d-6f133a1fbe7d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:50.279643 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030d709-64e7-4413-a21d-6f133a1fbe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:50.295076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4030d709-64e7-4413-a21d-6f133a1fbe7d None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:50.296290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4030d709-64e7-4413-a21d-6f133a1fbe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:50.358380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1321/2660] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:50.368485 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:50.369901 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:50.377263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:50.427341 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead21c7c-be7a-4ef4-9821-b155fb9ca7ea admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:50.431580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7736b5-3fe5-4286-9aff-16b24f3a8656 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:50.432215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7736b5-3fe5-4286-9aff-16b24f3a8656 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:50.432498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7736b5-3fe5-4286-9aff-16b24f3a8656 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:50.432774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7736b5-3fe5-4286-9aff-16b24f3a8656 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:50.445908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7736b5-3fe5-4286-9aff-16b24f3a8656 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:50.446231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030d709-64e7-4413-a21d-6f133a1fbe7d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:50.447844 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4030d709-64e7-4413-a21d-6f133a1fbe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:50.448315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7736b5-3fe5-4286-9aff-16b24f3a8656 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:50.460878 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030d709-64e7-4413-a21d-6f133a1fbe7d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:50.461264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1340/2661] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:50.466322 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:50.467448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:50.467800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:50.468092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] PATH_INFO: `/v3/projects/f6c6cae5e8724497a9b5bf14688a8c5c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:50.469635 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c6cae5e8724497a9b5bf14688a8c5c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:50.471647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:50.625670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c6cae5e8724497a9b5bf14688a8c5c', 'request_id': 'req-ad49149f-f97d-458b-9de2-2ae78e5bed8d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.631866 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c6cae5e8724497a9b5bf14688a8c5c', 'request_id': 'req-ad49149f-f97d-458b-9de2-2ae78e5bed8d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.637579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c6cae5e8724497a9b5bf14688a8c5c', 'request_id': 'req-ad49149f-f97d-458b-9de2-2ae78e5bed8d'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.729823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] The token cache is being invalidate because project f6c6cae5e8724497a9b5bf14688a8c5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:50.736747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.736747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:50.736747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1322/2662] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:50 2022] DELETE /identity/v3/projects/f6c6cae5e8724497a9b5bf14688a8c5c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:50.740626 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7736b5-3fe5-4286-9aff-16b24f3a8656 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:50.751110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7736b5-3fe5-4286-9aff-16b24f3a8656 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:50.817298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:50.818206 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:50.818487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:50.818762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] PATH_INFO: `/v3/users/b68c3025dc294fdaabd502236c446c93` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:50.819984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68c3025dc294fdaabd502236c446c93)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:50.822153 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:50.952830 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:50.954421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:50.974115 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49149f-f97d-458b-9de2-2ae78e5bed8d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:50.982567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:50.982567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:50.982567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:50.982567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] PATH_INFO: `/v3/projects/0bd351ab95cf4b3d92dd7de1135d7b4a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:50.983491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd351ab95cf4b3d92dd7de1135d7b4a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:50.985421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:50.992645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:51.081452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68c3025dc294fdaabd502236c446c93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.088977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68c3025dc294fdaabd502236c446c93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.100606 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68c3025dc294fdaabd502236c446c93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.101295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68c3025dc294fdaabd502236c446c93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.151575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd351ab95cf4b3d92dd7de1135d7b4a', 'request_id': 'req-7b55140c-77f7-44f4-965d-5bccb4ca901f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.156612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ce1425d-11ce-4151-b51e-bd187a51b064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68c3025dc294fdaabd502236c446c93'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.158642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd351ab95cf4b3d92dd7de1135d7b4a', 'request_id': 'req-7b55140c-77f7-44f4-965d-5bccb4ca901f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.165653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1341/2663] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:50 2022] DELETE /identity/v3/users/b68c3025dc294fdaabd502236c446c93 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:51.165887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd351ab95cf4b3d92dd7de1135d7b4a', 'request_id': 'req-7b55140c-77f7-44f4-965d-5bccb4ca901f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.235996 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] The token cache is being invalidate because project 0bd351ab95cf4b3d92dd7de1135d7b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:51.236288 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.236915 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-7b55140c-77f7-44f4-965d-5bccb4ca901f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.239096 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1323/2664] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:50 2022] DELETE /identity/v3/projects/0bd351ab95cf4b3d92dd7de1135d7b4a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:51.243996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72209-de29-4da0-9deb-ac5818b479fc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.244321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72209-de29-4da0-9deb-ac5818b479fc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.244608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72209-de29-4da0-9deb-ac5818b479fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.263599 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-43f72209-de29-4da0-9deb-ac5818b479fc None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:51.264964 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-43f72209-de29-4da0-9deb-ac5818b479fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:51.286591 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:51.325553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1342/2665] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:51.333233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931c007-a79e-4d6e-b259-32124913ebf0 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.333529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931c007-a79e-4d6e-b259-32124913ebf0 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.333811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931c007-a79e-4d6e-b259-32124913ebf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.352621 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-9931c007-a79e-4d6e-b259-32124913ebf0 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:51.353862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-9931c007-a79e-4d6e-b259-32124913ebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:51.362183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1324/2666] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:51.376837 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72209-de29-4da0-9deb-ac5818b479fc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:51.387330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72209-de29-4da0-9deb-ac5818b479fc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:51.392398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:51.393059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.393425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.393707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.394434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:51.395350 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:51.399988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:51.413796 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1343/2667] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:51.418165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931c007-a79e-4d6e-b259-32124913ebf0 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:51.427759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931c007-a79e-4d6e-b259-32124913ebf0 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:51.433032 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:51.433752 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.434021 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.434293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.435019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:51.436916 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:51.546870 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1325/2668] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:51 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:51.550859 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:51.561464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08be34-94b1-42e5-a075-6cf010e196f1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:51.615071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914848d-c130-4577-98fa-4d86a78950d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:51.615738 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914848d-c130-4577-98fa-4d86a78950d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.616020 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914848d-c130-4577-98fa-4d86a78950d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.616295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914848d-c130-4577-98fa-4d86a78950d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.616962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914848d-c130-4577-98fa-4d86a78950d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:51.619377 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6914848d-c130-4577-98fa-4d86a78950d1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:51.645541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:51.647298 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:51.660581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c592c-c209-4789-93a4-13361adb0f67 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:51.665488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:51.666389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.666808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.667097 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] PATH_INFO: `/v3/projects/8d92bb261ea94ab9a8b806e426d512d7` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.668347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d92bb261ea94ab9a8b806e426d512d7)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:51.670255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:51.716663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1344/2669] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:51 2022] POST /identity/v3/users => generated 302 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:51.720514 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914848d-c130-4577-98fa-4d86a78950d1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:51.730956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6914848d-c130-4577-98fa-4d86a78950d1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:51.735693 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c2b8c-cf13-4381-a853-2fa7214f8c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:51.736342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c2b8c-cf13-4381-a853-2fa7214f8c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.736630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c2b8c-cf13-4381-a853-2fa7214f8c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.736914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c2b8c-cf13-4381-a853-2fa7214f8c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.737721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c2b8c-cf13-4381-a853-2fa7214f8c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:51.739314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c2b8c-cf13-4381-a853-2fa7214f8c06 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:51.750564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1345/2670] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:51.754548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c2b8c-cf13-4381-a853-2fa7214f8c06 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:51.764186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c2b8c-cf13-4381-a853-2fa7214f8c06 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:51.768915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c00a4e-aca1-4f36-a954-98227d542de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:51.769580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c00a4e-aca1-4f36-a954-98227d542de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.769872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c00a4e-aca1-4f36-a954-98227d542de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.770156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c00a4e-aca1-4f36-a954-98227d542de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.770866 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c00a4e-aca1-4f36-a954-98227d542de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:51.772440 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c00a4e-aca1-4f36-a954-98227d542de9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:51.782663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1346/2671] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:51.786534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c00a4e-aca1-4f36-a954-98227d542de9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:51.797500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c00a4e-aca1-4f36-a954-98227d542de9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:51.806609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d92bb261ea94ab9a8b806e426d512d7', 'request_id': 'req-20ce9a7e-3822-4b9d-8990-398352893fc2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.813282 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d92bb261ea94ab9a8b806e426d512d7', 'request_id': 'req-20ce9a7e-3822-4b9d-8990-398352893fc2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.820443 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d92bb261ea94ab9a8b806e426d512d7', 'request_id': 'req-20ce9a7e-3822-4b9d-8990-398352893fc2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.854272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c191c-ce34-4267-a290-6202fbde10bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:51.855162 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c191c-ce34-4267-a290-6202fbde10bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.855443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c191c-ce34-4267-a290-6202fbde10bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.855724 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c191c-ce34-4267-a290-6202fbde10bf admin admin] PATH_INFO: `/v3/projects/823372072c9244ff8234a87f456400b7/users/ab8461461991413ab42518ed41f19372/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.869957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c191c-ce34-4267-a290-6202fbde10bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=823372072c9244ff8234a87f456400b7, user_id=ab8461461991413ab42518ed41f19372, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:51.872923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c191c-ce34-4267-a290-6202fbde10bf admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:51.890081 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] The token cache is being invalidate because project 8d92bb261ea94ab9a8b806e426d512d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:51.890380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.891042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-20ce9a7e-3822-4b9d-8990-398352893fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:51.892658 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1326/2672] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:51 2022] DELETE /identity/v3/projects/8d92bb261ea94ab9a8b806e426d512d7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:51.897232 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8c84-557a-4c01-9e97-75865bc0a46e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.897720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8c84-557a-4c01-9e97-75865bc0a46e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.898073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca8c84-557a-4c01-9e97-75865bc0a46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.916978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-15ca8c84-557a-4c01-9e97-75865bc0a46e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:51.918248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-15ca8c84-557a-4c01-9e97-75865bc0a46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:51.926386 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1327/2673] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:51.933921 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48332338-00b1-452f-a61f-4dd7f4cc4781 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.934216 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48332338-00b1-452f-a61f-4dd7f4cc4781 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.934524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48332338-00b1-452f-a61f-4dd7f4cc4781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.948656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1347/2674] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:51 2022] PUT /identity/v3/projects/823372072c9244ff8234a87f456400b7/users/ab8461461991413ab42518ed41f19372/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:51.954291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-48332338-00b1-452f-a61f-4dd7f4cc4781 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:51.955545 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-48332338-00b1-452f-a61f-4dd7f4cc4781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:51.960762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a962da-e125-45e8-a7fb-9af11cf0959d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:51.961630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a962da-e125-45e8-a7fb-9af11cf0959d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:51.961918 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a962da-e125-45e8-a7fb-9af11cf0959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:51.968648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:51.987774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c5a962da-e125-45e8-a7fb-9af11cf0959d None None] MFA Rules not processed for user `ab8461461991413ab42518ed41f19372`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:51.988968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5a962da-e125-45e8-a7fb-9af11cf0959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:51.996949 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:52.019246 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1328/2675] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:52.023373 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48332338-00b1-452f-a61f-4dd7f4cc4781 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.032785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48332338-00b1-452f-a61f-4dd7f4cc4781 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.037485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.038146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.038412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.038692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.039402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:52.039712 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.041333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.053520 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1329/2676] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:43:52.054288 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1348/2677] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:52.057989 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.058507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a962da-e125-45e8-a7fb-9af11cf0959d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.059838 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5a962da-e125-45e8-a7fb-9af11cf0959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:52.067920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23fe31-8896-4de5-a3a9-d977fe09c41e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.073592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a962da-e125-45e8-a7fb-9af11cf0959d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.076649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec23c28-f952-4582-aaf9-a8e5340b8848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.077345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec23c28-f952-4582-aaf9-a8e5340b8848 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.077634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec23c28-f952-4582-aaf9-a8e5340b8848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.077916 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec23c28-f952-4582-aaf9-a8e5340b8848 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.079399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec23c28-f952-4582-aaf9-a8e5340b8848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.080548 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec23c28-f952-4582-aaf9-a8e5340b8848 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.082039 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1332170-d451-4ddb-b03c-629f7c9d743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.082723 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332170-d451-4ddb-b03c-629f7c9d743e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.083009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332170-d451-4ddb-b03c-629f7c9d743e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.083283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332170-d451-4ddb-b03c-629f7c9d743e admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.084397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1332170-d451-4ddb-b03c-629f7c9d743e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.085990 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1332170-d451-4ddb-b03c-629f7c9d743e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.172041 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1349/2678] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:52 2022] POST /identity/v3/projects => generated 403 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:52.172978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1330/2679] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:52 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:52.175985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1332170-d451-4ddb-b03c-629f7c9d743e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.177122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec23c28-f952-4582-aaf9-a8e5340b8848 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.236782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec23c28-f952-4582-aaf9-a8e5340b8848 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.238179 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1332170-d451-4ddb-b03c-629f7c9d743e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.241538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305f953-f110-4efd-b928-2e3e47003158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.242734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305f953-f110-4efd-b928-2e3e47003158 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.243157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305f953-f110-4efd-b928-2e3e47003158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.243285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305f953-f110-4efd-b928-2e3e47003158 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.243946 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb97d3-2071-4133-92a5-4dbe22e9db3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.244080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305f953-f110-4efd-b928-2e3e47003158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.244919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb97d3-2071-4133-92a5-4dbe22e9db3e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.246153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305f953-f110-4efd-b928-2e3e47003158 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.246279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb97d3-2071-4133-92a5-4dbe22e9db3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.246585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb97d3-2071-4133-92a5-4dbe22e9db3e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.247287 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb97d3-2071-4133-92a5-4dbe22e9db3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.249193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb97d3-2071-4133-92a5-4dbe22e9db3e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.341945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1331/2680] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:52 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:52.344784 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305f953-f110-4efd-b928-2e3e47003158 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.350574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1350/2681] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:52 2022] POST /identity/v3/users => generated 308 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:52.351663 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb97d3-2071-4133-92a5-4dbe22e9db3e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.355174 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305f953-f110-4efd-b928-2e3e47003158 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.360747 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2886aa-f313-4080-bc59-741685559a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.361365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb97d3-2071-4133-92a5-4dbe22e9db3e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.361496 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2886aa-f313-4080-bc59-741685559a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.361710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2886aa-f313-4080-bc59-741685559a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.361994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2886aa-f313-4080-bc59-741685559a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.362770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2886aa-f313-4080-bc59-741685559a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.364396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2886aa-f313-4080-bc59-741685559a7e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.365930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324de42-104a-4b89-a1ec-a2d577b42346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.366573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324de42-104a-4b89-a1ec-a2d577b42346 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.366853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324de42-104a-4b89-a1ec-a2d577b42346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.367127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324de42-104a-4b89-a1ec-a2d577b42346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.368339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324de42-104a-4b89-a1ec-a2d577b42346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.369922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324de42-104a-4b89-a1ec-a2d577b42346 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.375772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1332/2682] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:52.379618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2886aa-f313-4080-bc59-741685559a7e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.383864 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1351/2683] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:52.389094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324de42-104a-4b89-a1ec-a2d577b42346 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.391145 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2886aa-f313-4080-bc59-741685559a7e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.396103 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c36856-99fb-4bbd-bcb1-253d5509e757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.396891 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36856-99fb-4bbd-bcb1-253d5509e757 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.397216 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36856-99fb-4bbd-bcb1-253d5509e757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.397507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c36856-99fb-4bbd-bcb1-253d5509e757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.398273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c36856-99fb-4bbd-bcb1-253d5509e757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.399911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c36856-99fb-4bbd-bcb1-253d5509e757 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.403625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324de42-104a-4b89-a1ec-a2d577b42346 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.409504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3f970-8dcf-40ed-9e30-f3ab750a279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.410403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3f970-8dcf-40ed-9e30-f3ab750a279e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.410832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3f970-8dcf-40ed-9e30-f3ab750a279e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.411294 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1333/2684] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:52.411433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3f970-8dcf-40ed-9e30-f3ab750a279e admin admin] PATH_INFO: `/v3/projects/e1974dc44f7d46b696abf345e6b8db5b/users/aaf3097c035a49fd9c512add5234c95f/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.415120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c36856-99fb-4bbd-bcb1-253d5509e757 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.424749 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c36856-99fb-4bbd-bcb1-253d5509e757 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.427070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3f970-8dcf-40ed-9e30-f3ab750a279e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1974dc44f7d46b696abf345e6b8db5b, user_id=aaf3097c035a49fd9c512add5234c95f, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.429485 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae979db-745b-45f7-a15c-5d24ff8cc8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.430468 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3f970-8dcf-40ed-9e30-f3ab750a279e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.430892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae979db-745b-45f7-a15c-5d24ff8cc8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.431182 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae979db-745b-45f7-a15c-5d24ff8cc8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.431461 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae979db-745b-45f7-a15c-5d24ff8cc8c0 admin admin] PATH_INFO: `/v3/projects/b2e24600795847d5b8c8ba41a3c7ff9f/users/b3b48c5807514e57a954766e543044c9/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.446860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae979db-745b-45f7-a15c-5d24ff8cc8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e24600795847d5b8c8ba41a3c7ff9f, user_id=b3b48c5807514e57a954766e543044c9, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.449624 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae979db-745b-45f7-a15c-5d24ff8cc8c0 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.535906 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1352/2685] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:52 2022] PUT /identity/v3/projects/e1974dc44f7d46b696abf345e6b8db5b/users/aaf3097c035a49fd9c512add5234c95f/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:52.541618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1334/2686] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:52 2022] PUT /identity/v3/projects/b2e24600795847d5b8c8ba41a3c7ff9f/users/b3b48c5807514e57a954766e543044c9/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:52.544504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3f970-8dcf-40ed-9e30-f3ab750a279e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.554454 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6aca8-a863-49a4-b179-484fc22f5eba None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.554796 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6aca8-a863-49a4-b179-484fc22f5eba None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.555611 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6aca8-a863-49a4-b179-484fc22f5eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.577870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:52.582057 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b8a6aca8-a863-49a4-b179-484fc22f5eba None None] MFA Rules not processed for user `b3b48c5807514e57a954766e543044c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:52.583268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b8a6aca8-a863-49a4-b179-484fc22f5eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:52.609746 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3f970-8dcf-40ed-9e30-f3ab750a279e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.624603 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:52.648238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1335/2687] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:52.652691 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6aca8-a863-49a4-b179-484fc22f5eba None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.668728 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f63870-8da1-4fa5-965d-c296d86262d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.669433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f63870-8da1-4fa5-965d-c296d86262d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.669716 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f63870-8da1-4fa5-965d-c296d86262d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.669997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f63870-8da1-4fa5-965d-c296d86262d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.670722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f63870-8da1-4fa5-965d-c296d86262d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.672318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f63870-8da1-4fa5-965d-c296d86262d7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.686239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1353/2688] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:52.687982 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f63870-8da1-4fa5-965d-c296d86262d7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.697187 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f63870-8da1-4fa5-965d-c296d86262d7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.702467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e281122d-359c-497a-afc9-494a2d874998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.703419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281122d-359c-497a-afc9-494a2d874998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.703710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281122d-359c-497a-afc9-494a2d874998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.703991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e281122d-359c-497a-afc9-494a2d874998 admin admin] PATH_INFO: `/v3/projects/e1974dc44f7d46b696abf345e6b8db5b/users/aaf3097c035a49fd9c512add5234c95f/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.706002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281122d-359c-497a-afc9-494a2d874998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1974dc44f7d46b696abf345e6b8db5b, user_id=aaf3097c035a49fd9c512add5234c95f, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.708790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e281122d-359c-497a-afc9-494a2d874998 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.716510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6aca8-a863-49a4-b179-484fc22f5eba None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.719248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1354/2689] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:52 2022] PUT /identity/v3/projects/e1974dc44f7d46b696abf345e6b8db5b/users/aaf3097c035a49fd9c512add5234c95f/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:52.730552 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177446d7-961e-4455-b29f-ece811e5ee2a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.730857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177446d7-961e-4455-b29f-ece811e5ee2a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.731145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177446d7-961e-4455-b29f-ece811e5ee2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.757522 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-177446d7-961e-4455-b29f-ece811e5ee2a None None] MFA Rules not processed for user `aaf3097c035a49fd9c512add5234c95f`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:52.759319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-177446d7-961e-4455-b29f-ece811e5ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:52.775868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e22585-2bd7-4af1-9cd6-14455f291a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.776530 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e22585-2bd7-4af1-9cd6-14455f291a5e admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.776812 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e22585-2bd7-4af1-9cd6-14455f291a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.777089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e22585-2bd7-4af1-9cd6-14455f291a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.777793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e22585-2bd7-4af1-9cd6-14455f291a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.779725 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e22585-2bd7-4af1-9cd6-14455f291a5e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.829382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1355/2690] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:52.840504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177446d7-961e-4455-b29f-ece811e5ee2a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.846285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-177446d7-961e-4455-b29f-ece811e5ee2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:52.890857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:52.902412 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177446d7-961e-4455-b29f-ece811e5ee2a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:52.906193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511591d-0dc4-4dad-b68d-218891520675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:52.906842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511591d-0dc4-4dad-b68d-218891520675 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:52.907126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511591d-0dc4-4dad-b68d-218891520675 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:52.907404 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511591d-0dc4-4dad-b68d-218891520675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:52.917815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511591d-0dc4-4dad-b68d-218891520675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:52.920922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511591d-0dc4-4dad-b68d-218891520675 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:52.921053 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1336/2691] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:52 2022] POST /identity/v3/projects => generated 387 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:52.924413 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e22585-2bd7-4af1-9cd6-14455f291a5e admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:52.929060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1356/2692] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:52.995928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e22585-2bd7-4af1-9cd6-14455f291a5e admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:53.000642 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d1e36-8b90-4f14-b642-760407927a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:53.001341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1e36-8b90-4f14-b642-760407927a88 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.001744 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1e36-8b90-4f14-b642-760407927a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.002026 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1e36-8b90-4f14-b642-760407927a88 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.002708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d1e36-8b90-4f14-b642-760407927a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:53.004616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d1e36-8b90-4f14-b642-760407927a88 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:53.061686 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511591d-0dc4-4dad-b68d-218891520675 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:53.072788 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511591d-0dc4-4dad-b68d-218891520675 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:53.077387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:53.078229 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.078509 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.078783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] PATH_INFO: `/v3/users/ab8461461991413ab42518ed41f19372` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.079938 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab8461461991413ab42518ed41f19372)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:53.081861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:53.088445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1337/2693] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:52 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:53.092199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d1e36-8b90-4f14-b642-760407927a88 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:53.097243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:53.102258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d1e36-8b90-4f14-b642-760407927a88 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:53.106998 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a51e1-db30-44f9-8d3b-dd188e206e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:53.107699 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a51e1-db30-44f9-8d3b-dd188e206e81 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.107967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a51e1-db30-44f9-8d3b-dd188e206e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.108243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a51e1-db30-44f9-8d3b-dd188e206e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.108959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a51e1-db30-44f9-8d3b-dd188e206e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:53.110607 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a51e1-db30-44f9-8d3b-dd188e206e81 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:53.122009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1338/2694] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:53.126043 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a51e1-db30-44f9-8d3b-dd188e206e81 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:53.136016 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a51e1-db30-44f9-8d3b-dd188e206e81 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:53.140795 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6cc23-93e5-4df8-8e3a-597b7e78f28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:53.141713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6cc23-93e5-4df8-8e3a-597b7e78f28f admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.142000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6cc23-93e5-4df8-8e3a-597b7e78f28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.142278 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6cc23-93e5-4df8-8e3a-597b7e78f28f admin admin] PATH_INFO: `/v3/projects/7496598eb4c0417daed003a2ae092810/users/73c366ea505a473f98b2243f3ffb345a/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.156365 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6cc23-93e5-4df8-8e3a-597b7e78f28f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7496598eb4c0417daed003a2ae092810, user_id=73c366ea505a473f98b2243f3ffb345a, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:53.159069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6cc23-93e5-4df8-8e3a-597b7e78f28f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:53.282932 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1339/2695] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:53 2022] PUT /identity/v3/projects/7496598eb4c0417daed003a2ae092810/users/73c366ea505a473f98b2243f3ffb345a/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:53.287288 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6cc23-93e5-4df8-8e3a-597b7e78f28f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:53.291334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab8461461991413ab42518ed41f19372'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:53.298671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab8461461991413ab42518ed41f19372'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:53.311499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab8461461991413ab42518ed41f19372'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:53.312208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab8461461991413ab42518ed41f19372'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:53.348193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6cc23-93e5-4df8-8e3a-597b7e78f28f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:53.353458 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a8201-2c16-4ef8-8182-c982e8c9c757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:53.354113 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a8201-2c16-4ef8-8182-c982e8c9c757 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.354397 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a8201-2c16-4ef8-8182-c982e8c9c757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.354670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a8201-2c16-4ef8-8182-c982e8c9c757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.355383 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a8201-2c16-4ef8-8182-c982e8c9c757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:53.356973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a8201-2c16-4ef8-8182-c982e8c9c757 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:53.366763 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1340/2696] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:53.371736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a8201-2c16-4ef8-8182-c982e8c9c757 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:53.381325 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a8201-2c16-4ef8-8182-c982e8c9c757 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:53.384670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4bce4704-b495-4ac0-a332-a63b1e166192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab8461461991413ab42518ed41f19372'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:53.385898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075bfac-48ee-4f44-8fdf-0a2454729107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:53.389438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bfac-48ee-4f44-8fdf-0a2454729107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.389438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bfac-48ee-4f44-8fdf-0a2454729107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.389438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075bfac-48ee-4f44-8fdf-0a2454729107 admin admin] PATH_INFO: `/v3/projects/7496598eb4c0417daed003a2ae092810/users/73c366ea505a473f98b2243f3ffb345a/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.389438 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075bfac-48ee-4f44-8fdf-0a2454729107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7496598eb4c0417daed003a2ae092810, user_id=73c366ea505a473f98b2243f3ffb345a, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:53.391373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1357/2697] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:53 2022] DELETE /identity/v3/users/ab8461461991413ab42518ed41f19372 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:53.393652 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075bfac-48ee-4f44-8fdf-0a2454729107 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:53.395692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df6caa-0d17-4e1d-ace4-0d37f8c0a660 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.396081 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df6caa-0d17-4e1d-ace4-0d37f8c0a660 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.396370 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df6caa-0d17-4e1d-ace4-0d37f8c0a660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.404719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1341/2698] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:53 2022] PUT /identity/v3/projects/7496598eb4c0417daed003a2ae092810/users/73c366ea505a473f98b2243f3ffb345a/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:53.413512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a1df6caa-0d17-4e1d-ace4-0d37f8c0a660 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:53.418144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a1df6caa-0d17-4e1d-ace4-0d37f8c0a660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:53.418305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea8c-04f9-4d70-91fa-b090282ad04f None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.418305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea8c-04f9-4d70-91fa-b090282ad04f None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.418305 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957ea8c-04f9-4d70-91fa-b090282ad04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.448401 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3957ea8c-04f9-4d70-91fa-b090282ad04f None None] MFA Rules not processed for user `73c366ea505a473f98b2243f3ffb345a`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:53.450054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3957ea8c-04f9-4d70-91fa-b090282ad04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:53.486353 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1358/2699] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:53.496826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df6caa-0d17-4e1d-ace4-0d37f8c0a660 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:53.498611 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a1df6caa-0d17-4e1d-ace4-0d37f8c0a660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:53.535769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1342/2700] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:53.550793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957ea8c-04f9-4d70-91fa-b090282ad04f None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:53.552151 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3957ea8c-04f9-4d70-91fa-b090282ad04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:53.558919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df6caa-0d17-4e1d-ace4-0d37f8c0a660 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:53.564893 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:53.566023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.566475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.567075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.582205 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:53.583694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:53.590458 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:53.595453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:53.595879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1359/2701] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:53.613089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957ea8c-04f9-4d70-91fa-b090282ad04f None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:53.616916 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc3163-f1fe-460c-93dd-fedda6b8649a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:53.618184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3163-f1fe-460c-93dd-fedda6b8649a service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:53.618542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3163-f1fe-460c-93dd-fedda6b8649a service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:53.618825 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3163-f1fe-460c-93dd-fedda6b8649a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:53.628544 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc3163-f1fe-460c-93dd-fedda6b8649a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:53.630046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc3163-f1fe-460c-93dd-fedda6b8649a service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:53.640739 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1343/2702] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:54.048710 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:54.050263 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:54.063309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9453ee4-48ef-44c4-b4c4-87583a3a8acb service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:54.068740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:54.069781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:54.070192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:54.074747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] PATH_INFO: `/v3/projects/823372072c9244ff8234a87f456400b7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:54.074747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=823372072c9244ff8234a87f456400b7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:54.074747 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:54.248132 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823372072c9244ff8234a87f456400b7', 'request_id': 'req-e2801a0f-3d11-4f91-83b5-176909aa6af9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.254488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823372072c9244ff8234a87f456400b7', 'request_id': 'req-e2801a0f-3d11-4f91-83b5-176909aa6af9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.262065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823372072c9244ff8234a87f456400b7', 'request_id': 'req-e2801a0f-3d11-4f91-83b5-176909aa6af9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.398398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] The token cache is being invalidate because project 823372072c9244ff8234a87f456400b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:54.398851 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.399526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.401700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1360/2703] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:54 2022] DELETE /identity/v3/projects/823372072c9244ff8234a87f456400b7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:54.405470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc3163-f1fe-460c-93dd-fedda6b8649a service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:54.416196 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc3163-f1fe-460c-93dd-fedda6b8649a service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:54.472713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:54.473633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:54.473924 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:54.474202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] PATH_INFO: `/v3/users/aaf3097c035a49fd9c512add5234c95f` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:54.475391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf3097c035a49fd9c512add5234c95f)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:54.477353 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:54.635347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf3097c035a49fd9c512add5234c95f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.647412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf3097c035a49fd9c512add5234c95f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.648119 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf3097c035a49fd9c512add5234c95f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.648738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf3097c035a49fd9c512add5234c95f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.763833 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf3097c035a49fd9c512add5234c95f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:54.776638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1344/2704] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:54 2022] DELETE /identity/v3/users/aaf3097c035a49fd9c512add5234c95f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:54.878002 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:54.879415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:54.894824 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2801a0f-3d11-4f91-83b5-176909aa6af9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:54.950679 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:54.951612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:54.951897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:54.952194 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] PATH_INFO: `/v3/users/b3b48c5807514e57a954766e543044c9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:54.953415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3b48c5807514e57a954766e543044c9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:54.955343 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:55.164928 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3b48c5807514e57a954766e543044c9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.171647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3b48c5807514e57a954766e543044c9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.183684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3b48c5807514e57a954766e543044c9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.184517 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3b48c5807514e57a954766e543044c9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.219033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:55.219195 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:55.236076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760bd1c-1952-4497-ab52-277c1298bd14 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:55.240868 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:55.241769 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.242048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.242434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] PATH_INFO: `/v3/projects/e1974dc44f7d46b696abf345e6b8db5b` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.243703 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1974dc44f7d46b696abf345e6b8db5b)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:55.248152 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:55.295388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ce6d7c50-d17d-4c30-a69e-b848df79bb1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3b48c5807514e57a954766e543044c9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.304484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1361/2705] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:54 2022] DELETE /identity/v3/users/b3b48c5807514e57a954766e543044c9 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:55.309476 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41e385-5e1f-44c5-8a92-53b26ef24a24 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.309846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41e385-5e1f-44c5-8a92-53b26ef24a24 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.310197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41e385-5e1f-44c5-8a92-53b26ef24a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.316260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:55.331049 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7d41e385-5e1f-44c5-8a92-53b26ef24a24 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:55.332277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7d41e385-5e1f-44c5-8a92-53b26ef24a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:55.391633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1974dc44f7d46b696abf345e6b8db5b', 'request_id': 'req-2a285c8e-8528-4575-bfd1-4ea531e16517'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.393566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:55.394664 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1362/2706] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:55.398199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1974dc44f7d46b696abf345e6b8db5b', 'request_id': 'req-2a285c8e-8528-4575-bfd1-4ea531e16517'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.404626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1974dc44f7d46b696abf345e6b8db5b', 'request_id': 'req-2a285c8e-8528-4575-bfd1-4ea531e16517'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.405029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41e385-5e1f-44c5-8a92-53b26ef24a24 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:55.406411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7d41e385-5e1f-44c5-8a92-53b26ef24a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:55.460223 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41e385-5e1f-44c5-8a92-53b26ef24a24 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:55.487507 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] The token cache is being invalidate because project e1974dc44f7d46b696abf345e6b8db5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:55.488058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.488474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-2a285c8e-8528-4575-bfd1-4ea531e16517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:55.490070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1345/2707] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:55 2022] DELETE /identity/v3/projects/e1974dc44f7d46b696abf345e6b8db5b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:55.494842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53374d4-523d-4075-a2d8-bf50e5ee6650 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.495471 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53374d4-523d-4075-a2d8-bf50e5ee6650 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.495754 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53374d4-523d-4075-a2d8-bf50e5ee6650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.513029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:55.513706 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.513991 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.514233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c53374d4-523d-4075-a2d8-bf50e5ee6650 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:55.514339 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.515455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c53374d4-523d-4075-a2d8-bf50e5ee6650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:55.515998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:55.573611 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1346/2708] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:55.580955 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981e043-60a2-4d4b-a506-4e4154310032 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.581547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981e043-60a2-4d4b-a506-4e4154310032 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.582256 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981e043-60a2-4d4b-a506-4e4154310032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.582395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:55.583823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:55.596298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1363/2709] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:55.602283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-3981e043-60a2-4d4b-a506-4e4154310032 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:55.615865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3981e043-60a2-4d4b-a506-4e4154310032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:55.615865 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1347/2710] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:55.616222 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:55.625994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1f67f-6ae5-4420-9b22-48807f97856f service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:55.630792 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:55.631452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.631742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.632018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.633251 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:55.633627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:55.635290 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:55.637940 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:55.647165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1364/2711] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:43:55.651584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981e043-60a2-4d4b-a506-4e4154310032 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:55.660994 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981e043-60a2-4d4b-a506-4e4154310032 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:55.666255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84e213-dd30-497d-b25d-97638a5b15cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:55.666902 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84e213-dd30-497d-b25d-97638a5b15cc admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.667183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84e213-dd30-497d-b25d-97638a5b15cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.667457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84e213-dd30-497d-b25d-97638a5b15cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.668138 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84e213-dd30-497d-b25d-97638a5b15cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:55.670176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84e213-dd30-497d-b25d-97638a5b15cc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:55.806052 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1348/2712] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:55 2022] POST /identity/v3/projects => generated 403 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:55.810086 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:55.820159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d07098-2f65-4ca8-b20f-78a16b484275 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:55.878233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b9e67-8a2e-43cc-8c22-f4fde9860ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:55.879062 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9e67-8a2e-43cc-8c22-f4fde9860ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.879417 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9e67-8a2e-43cc-8c22-f4fde9860ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.879700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b9e67-8a2e-43cc-8c22-f4fde9860ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.880383 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b9e67-8a2e-43cc-8c22-f4fde9860ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:55.882464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b9e67-8a2e-43cc-8c22-f4fde9860ff9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:55.958936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1365/2713] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:55 2022] POST /identity/v3/users => generated 302 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:55.963490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84e213-dd30-497d-b25d-97638a5b15cc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:55.974299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84e213-dd30-497d-b25d-97638a5b15cc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:55.980060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ff9dc-6505-4bb6-87b1-c45aa7fcc180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:55.980745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ff9dc-6505-4bb6-87b1-c45aa7fcc180 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:55.981035 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ff9dc-6505-4bb6-87b1-c45aa7fcc180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:55.981351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ff9dc-6505-4bb6-87b1-c45aa7fcc180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:55.982080 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ff9dc-6505-4bb6-87b1-c45aa7fcc180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:55.984242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ff9dc-6505-4bb6-87b1-c45aa7fcc180 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:55.994664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1349/2714] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:55.999257 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b9e67-8a2e-43cc-8c22-f4fde9860ff9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:56.009643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b9e67-8a2e-43cc-8c22-f4fde9860ff9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:56.014273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:56.014929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:56.015213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:56.015490 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:56.016232 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:56.017899 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:56.028456 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1366/2715] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:56.032745 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ff9dc-6505-4bb6-87b1-c45aa7fcc180 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:56.043388 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ff9dc-6505-4bb6-87b1-c45aa7fcc180 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:56.048019 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16724363-76b2-4b67-b536-07a67a11a343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:56.048914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724363-76b2-4b67-b536-07a67a11a343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:56.049247 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724363-76b2-4b67-b536-07a67a11a343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:56.059199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724363-76b2-4b67-b536-07a67a11a343 admin admin] PATH_INFO: `/v3/projects/92a6848c84c84a84847deda84133204c/users/73481e3bc60148479b2c5b36c2da5fb7/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:56.064172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16724363-76b2-4b67-b536-07a67a11a343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a6848c84c84a84847deda84133204c, user_id=73481e3bc60148479b2c5b36c2da5fb7, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:56.066908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16724363-76b2-4b67-b536-07a67a11a343 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:56.086713 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:56.088544 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:56.105031 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91966b6-9656-4c44-a6a5-4624eea519f5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:56.110018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:56.110897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:56.111161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:56.111442 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] PATH_INFO: `/v3/projects/b2e24600795847d5b8c8ba41a3c7ff9f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:56.112608 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e24600795847d5b8c8ba41a3c7ff9f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:56.116810 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:56.170646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1350/2716] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:56 2022] PUT /identity/v3/projects/92a6848c84c84a84847deda84133204c/users/73481e3bc60148479b2c5b36c2da5fb7/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:56.175428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439050b-e861-49bf-8977-252e0ea8fa6b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:56.175716 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439050b-e861-49bf-8977-252e0ea8fa6b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:56.175991 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439050b-e861-49bf-8977-252e0ea8fa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:56.192326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-d439050b-e861-49bf-8977-252e0ea8fa6b None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:56.202583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d439050b-e861-49bf-8977-252e0ea8fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:56.251307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1351/2717] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:56.261063 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439050b-e861-49bf-8977-252e0ea8fa6b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:56.262434 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-d439050b-e861-49bf-8977-252e0ea8fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:56.274358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e24600795847d5b8c8ba41a3c7ff9f', 'request_id': 'req-b5f224ad-b480-4430-8efa-5e2124c5608f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.281725 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e24600795847d5b8c8ba41a3c7ff9f', 'request_id': 'req-b5f224ad-b480-4430-8efa-5e2124c5608f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.282605 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:56.287577 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e24600795847d5b8c8ba41a3c7ff9f', 'request_id': 'req-b5f224ad-b480-4430-8efa-5e2124c5608f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.318564 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439050b-e861-49bf-8977-252e0ea8fa6b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:56.365157 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de3398-3d6d-4604-999e-9eedaacb7403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:56.365801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de3398-3d6d-4604-999e-9eedaacb7403 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:56.366079 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de3398-3d6d-4604-999e-9eedaacb7403 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:56.366351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de3398-3d6d-4604-999e-9eedaacb7403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:56.375808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] The token cache is being invalidate because project b2e24600795847d5b8c8ba41a3c7ff9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:56.376127 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.376773 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.378447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1367/2718] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:56 2022] DELETE /identity/v3/projects/b2e24600795847d5b8c8ba41a3c7ff9f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:56.382557 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:56.385415 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:56.429834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de3398-3d6d-4604-999e-9eedaacb7403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:56.431324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de3398-3d6d-4604-999e-9eedaacb7403 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:56.440783 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1352/2719] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:56.457144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:56.461807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f224ad-b480-4430-8efa-5e2124c5608f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:56.466844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:56.467558 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:56.467840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:56.468113 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] PATH_INFO: `/v3/users/73c366ea505a473f98b2243f3ffb345a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:56.470284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c366ea505a473f98b2243f3ffb345a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:56.472267 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:56.633381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c366ea505a473f98b2243f3ffb345a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.640216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c366ea505a473f98b2243f3ffb345a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.653617 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c366ea505a473f98b2243f3ffb345a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.654605 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c366ea505a473f98b2243f3ffb345a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.746790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fe9a7df1-e905-420e-ba29-cc4ba821e2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c366ea505a473f98b2243f3ffb345a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:56.754071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1368/2720] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:43:56 2022] DELETE /identity/v3/users/73c366ea505a473f98b2243f3ffb345a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:57.263467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de3398-3d6d-4604-999e-9eedaacb7403 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:57.274591 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de3398-3d6d-4604-999e-9eedaacb7403 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:57.333547 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:57.334467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:57.339200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:57.339200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] PATH_INFO: `/v3/projects/7496598eb4c0417daed003a2ae092810` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:57.339200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7496598eb4c0417daed003a2ae092810)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:57.339200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:57.468893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:57.485094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7496598eb4c0417daed003a2ae092810', 'request_id': 'req-5cf3041c-5775-4911-8ae4-6dae99b34986'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:57.491153 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7496598eb4c0417daed003a2ae092810', 'request_id': 'req-5cf3041c-5775-4911-8ae4-6dae99b34986'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:57.497798 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7496598eb4c0417daed003a2ae092810', 'request_id': 'req-5cf3041c-5775-4911-8ae4-6dae99b34986'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:57.570189 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] The token cache is being invalidate because project 7496598eb4c0417daed003a2ae092810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:43:57.571142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:57.571886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-5cf3041c-5775-4911-8ae4-6dae99b34986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:43:57.575309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1353/2721] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:43:57 2022] DELETE /identity/v3/projects/7496598eb4c0417daed003a2ae092810 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:57.579635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce705722-24b9-426d-bcf2-0b15393e808f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:57.580143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce705722-24b9-426d-bcf2-0b15393e808f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:57.580503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce705722-24b9-426d-bcf2-0b15393e808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:57.600156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ce705722-24b9-426d-bcf2-0b15393e808f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:57.601533 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ce705722-24b9-426d-bcf2-0b15393e808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:57.666145 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1369/2722] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:57.674287 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983ed69-52c3-459b-bd97-cb958bef6719 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:57.674578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983ed69-52c3-459b-bd97-cb958bef6719 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:57.674883 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983ed69-52c3-459b-bd97-cb958bef6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:57.693678 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-c983ed69-52c3-459b-bd97-cb958bef6719 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:57.694871 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c983ed69-52c3-459b-bd97-cb958bef6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:57.703389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1354/2723] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:57.707537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce705722-24b9-426d-bcf2-0b15393e808f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:57.718518 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce705722-24b9-426d-bcf2-0b15393e808f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:57.723237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:57.723819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:57.724103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:57.724381 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:57.725146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:43:57.725445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:57.727722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:57.739409 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1370/2724] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:43:57.744362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983ed69-52c3-459b-bd97-cb958bef6719 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:57.755142 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983ed69-52c3-459b-bd97-cb958bef6719 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:57.759821 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7123d62-6908-4c97-a98a-12b20ff4ca7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:57.760460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7123d62-6908-4c97-a98a-12b20ff4ca7d admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:57.767208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7123d62-6908-4c97-a98a-12b20ff4ca7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:57.767208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7123d62-6908-4c97-a98a-12b20ff4ca7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:57.767208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7123d62-6908-4c97-a98a-12b20ff4ca7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:57.767208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7123d62-6908-4c97-a98a-12b20ff4ca7d admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:57.873259 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:57.874972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:57.904552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1355/2725] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:57 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:57.908594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7123d62-6908-4c97-a98a-12b20ff4ca7d admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:57.961443 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d903d7-44c9-4d49-ac11-eb596acca29f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:57.966531 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:57.986780 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7123d62-6908-4c97-a98a-12b20ff4ca7d admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:57.993790 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb652c-98a4-4f38-9db7-54c2490b9b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:57.995350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb652c-98a4-4f38-9db7-54c2490b9b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:57.996137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb652c-98a4-4f38-9db7-54c2490b9b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:57.996427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb652c-98a4-4f38-9db7-54c2490b9b8a admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:57.997304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb652c-98a4-4f38-9db7-54c2490b9b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:57.999265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb652c-98a4-4f38-9db7-54c2490b9b8a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.032845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b03c8-d3c2-4b5b-9428-722d53d52dd5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.033887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b03c8-d3c2-4b5b-9428-722d53d52dd5 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.034182 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b03c8-d3c2-4b5b-9428-722d53d52dd5 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.034467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b03c8-d3c2-4b5b-9428-722d53d52dd5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.036368 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-785b03c8-d3c2-4b5b-9428-722d53d52dd5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:58.048769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b03c8-d3c2-4b5b-9428-722d53d52dd5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.050393 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b03c8-d3c2-4b5b-9428-722d53d52dd5 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.061272 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1371/2726] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:43:58.062639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252f1b-f617-448c-8015-a6f0daa23294 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.062934 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252f1b-f617-448c-8015-a6f0daa23294 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.063215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252f1b-f617-448c-8015-a6f0daa23294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.090905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-22252f1b-f617-448c-8015-a6f0daa23294 None None] MFA Rules not processed for user `73481e3bc60148479b2c5b36c2da5fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:58.092102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-22252f1b-f617-448c-8015-a6f0daa23294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:58.097979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1356/2727] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:57 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:58.099313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb652c-98a4-4f38-9db7-54c2490b9b8a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.111477 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb652c-98a4-4f38-9db7-54c2490b9b8a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.116404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b35b9-536c-45f9-ad98-1122ddb26155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.117073 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b35b9-536c-45f9-ad98-1122ddb26155 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.117381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b35b9-536c-45f9-ad98-1122ddb26155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.117660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b35b9-536c-45f9-ad98-1122ddb26155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.118421 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b35b9-536c-45f9-ad98-1122ddb26155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.120054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b35b9-536c-45f9-ad98-1122ddb26155 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.130468 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1357/2728] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:58.134445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b35b9-536c-45f9-ad98-1122ddb26155 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.144116 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b35b9-536c-45f9-ad98-1122ddb26155 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.149333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36f594-a992-46d0-bf04-8c98f1c0bcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.149974 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36f594-a992-46d0-bf04-8c98f1c0bcfb admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.150264 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36f594-a992-46d0-bf04-8c98f1c0bcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.152351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36f594-a992-46d0-bf04-8c98f1c0bcfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.152351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36f594-a992-46d0-bf04-8c98f1c0bcfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.152811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36f594-a992-46d0-bf04-8c98f1c0bcfb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.159164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1372/2729] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:58.166983 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1358/2730] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:58.167157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22252f1b-f617-448c-8015-a6f0daa23294 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.167157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-22252f1b-f617-448c-8015-a6f0daa23294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:58.168183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36f594-a992-46d0-bf04-8c98f1c0bcfb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.178037 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36f594-a992-46d0-bf04-8c98f1c0bcfb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.179582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22252f1b-f617-448c-8015-a6f0daa23294 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.182628 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4ec99-a408-423f-9e8c-ebf015b78082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.183495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ec99-a408-423f-9e8c-ebf015b78082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.183774 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ec99-a408-423f-9e8c-ebf015b78082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.184049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ec99-a408-423f-9e8c-ebf015b78082 admin admin] PATH_INFO: `/v3/projects/77ce20a4fca84c419e1a561027d784b3/users/9bd5ccdeb434457bac9560aa5eca73e1/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.184321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca37c1-5f09-49d4-a39c-92fd0caee7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.185013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca37c1-5f09-49d4-a39c-92fd0caee7fa admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.185288 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca37c1-5f09-49d4-a39c-92fd0caee7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.185573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca37c1-5f09-49d4-a39c-92fd0caee7fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.186249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca37c1-5f09-49d4-a39c-92fd0caee7fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.188164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca37c1-5f09-49d4-a39c-92fd0caee7fa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.198860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4ec99-a408-423f-9e8c-ebf015b78082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ce20a4fca84c419e1a561027d784b3, user_id=9bd5ccdeb434457bac9560aa5eca73e1, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.201541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4ec99-a408-423f-9e8c-ebf015b78082 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.296043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1373/2731] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:58 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:58.300445 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca37c1-5f09-49d4-a39c-92fd0caee7fa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.306540 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1359/2732] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:58 2022] PUT /identity/v3/projects/77ce20a4fca84c419e1a561027d784b3/users/9bd5ccdeb434457bac9560aa5eca73e1/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:58.315928 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4869a-d9bd-44b7-b413-898a5e703c09 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.316223 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4869a-d9bd-44b7-b413-898a5e703c09 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.316505 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4869a-d9bd-44b7-b413-898a5e703c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.342626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-32b4869a-d9bd-44b7-b413-898a5e703c09 None None] MFA Rules not processed for user `9bd5ccdeb434457bac9560aa5eca73e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:58.343823 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-32b4869a-d9bd-44b7-b413-898a5e703c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:58.361672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:58.367890 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca37c1-5f09-49d4-a39c-92fd0caee7fa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.376894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:58.411302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1360/2733] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:43:58.416085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4869a-d9bd-44b7-b413-898a5e703c09 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.428396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b663674-fa29-4022-9bce-67b0db56e522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.430430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b663674-fa29-4022-9bce-67b0db56e522 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.430598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b663674-fa29-4022-9bce-67b0db56e522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.430790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b663674-fa29-4022-9bce-67b0db56e522 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.435362 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b663674-fa29-4022-9bce-67b0db56e522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.437927 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b663674-fa29-4022-9bce-67b0db56e522 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.485698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4869a-d9bd-44b7-b413-898a5e703c09 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.490286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918f0b9-36b0-40dc-8c38-f40785d9aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.490937 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918f0b9-36b0-40dc-8c38-f40785d9aed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.491219 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918f0b9-36b0-40dc-8c38-f40785d9aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.491491 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918f0b9-36b0-40dc-8c38-f40785d9aed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.492788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918f0b9-36b0-40dc-8c38-f40785d9aed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.494719 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918f0b9-36b0-40dc-8c38-f40785d9aed7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.553052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1374/2734] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:58 2022] POST /identity/v3/users => generated 309 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:43:58.553052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b663674-fa29-4022-9bce-67b0db56e522 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.560714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b663674-fa29-4022-9bce-67b0db56e522 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.565426 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75bdad-14e2-4017-b31c-fc102cc43262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.566077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75bdad-14e2-4017-b31c-fc102cc43262 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.566367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75bdad-14e2-4017-b31c-fc102cc43262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.566647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75bdad-14e2-4017-b31c-fc102cc43262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.567373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75bdad-14e2-4017-b31c-fc102cc43262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.568956 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75bdad-14e2-4017-b31c-fc102cc43262 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.579777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1375/2735] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:58.590465 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75bdad-14e2-4017-b31c-fc102cc43262 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.600040 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75bdad-14e2-4017-b31c-fc102cc43262 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.605209 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede41965-96c0-4a3b-897c-ad47c8244c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.606055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede41965-96c0-4a3b-897c-ad47c8244c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.606340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede41965-96c0-4a3b-897c-ad47c8244c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.606640 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede41965-96c0-4a3b-897c-ad47c8244c86 admin admin] PATH_INFO: `/v3/projects/801920a2d9dd4e3f82928cef7f94061b/users/045a8e388a02463aada268462f3f5905/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.622124 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede41965-96c0-4a3b-897c-ad47c8244c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801920a2d9dd4e3f82928cef7f94061b, user_id=045a8e388a02463aada268462f3f5905, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.624766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede41965-96c0-4a3b-897c-ad47c8244c86 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.669887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1361/2736] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:43:58 2022] POST /identity/v3/projects => generated 383 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:58.673841 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918f0b9-36b0-40dc-8c38-f40785d9aed7 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.741228 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918f0b9-36b0-40dc-8c38-f40785d9aed7 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.746908 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929eee09-c799-480d-be0b-1494f6f12f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.747569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eee09-c799-480d-be0b-1494f6f12f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.747855 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eee09-c799-480d-be0b-1494f6f12f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.748134 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eee09-c799-480d-be0b-1494f6f12f3f admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.748843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eee09-c799-480d-be0b-1494f6f12f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.750826 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eee09-c799-480d-be0b-1494f6f12f3f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.768985 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1376/2737] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:58 2022] PUT /identity/v3/projects/801920a2d9dd4e3f82928cef7f94061b/users/045a8e388a02463aada268462f3f5905/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:58.775198 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede41965-96c0-4a3b-897c-ad47c8244c86 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.835178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede41965-96c0-4a3b-897c-ad47c8244c86 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.840255 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de8062-ca8c-4136-b69f-374413b16f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.844114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8062-ca8c-4136-b69f-374413b16f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.844114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8062-ca8c-4136-b69f-374413b16f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.844114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8062-ca8c-4136-b69f-374413b16f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.844114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de8062-ca8c-4136-b69f-374413b16f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.844114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de8062-ca8c-4136-b69f-374413b16f23 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.853919 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1377/2738] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:58.858148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de8062-ca8c-4136-b69f-374413b16f23 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.869227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de8062-ca8c-4136-b69f-374413b16f23 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.874852 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1362/2739] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:43:58 2022] POST /identity/v3/users => generated 298 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:43:58.874995 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560704aa-e332-4cff-a164-c4a653a11218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.878402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560704aa-e332-4cff-a164-c4a653a11218 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.878402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560704aa-e332-4cff-a164-c4a653a11218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.878402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560704aa-e332-4cff-a164-c4a653a11218 admin admin] PATH_INFO: `/v3/projects/801920a2d9dd4e3f82928cef7f94061b/users/045a8e388a02463aada268462f3f5905/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.878402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560704aa-e332-4cff-a164-c4a653a11218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801920a2d9dd4e3f82928cef7f94061b, user_id=045a8e388a02463aada268462f3f5905, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.881093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560704aa-e332-4cff-a164-c4a653a11218 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.881263 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929eee09-c799-480d-be0b-1494f6f12f3f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.890913 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929eee09-c799-480d-be0b-1494f6f12f3f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.891752 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1378/2740] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:58 2022] PUT /identity/v3/projects/801920a2d9dd4e3f82928cef7f94061b/users/045a8e388a02463aada268462f3f5905/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:43:58.930313 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:58.955497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf1b18-f00f-45a1-ac28-9b214551afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.956146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf1b18-f00f-45a1-ac28-9b214551afa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.956428 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf1b18-f00f-45a1-ac28-9b214551afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.956706 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf1b18-f00f-45a1-ac28-9b214551afa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:58.957465 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf1b18-f00f-45a1-ac28-9b214551afa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:58.959060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf1b18-f00f-45a1-ac28-9b214551afa3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:58.973141 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1363/2741] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:43:58.974367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560704aa-e332-4cff-a164-c4a653a11218 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:58.986193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560704aa-e332-4cff-a164-c4a653a11218 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:58.991515 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbb008-750d-45d8-b363-5161dededce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:58.992754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbb008-750d-45d8-b363-5161dededce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:58.993140 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbb008-750d-45d8-b363-5161dededce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:58.993551 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fbb008-750d-45d8-b363-5161dededce4 admin admin] PATH_INFO: `/v3/projects/90a3606789f34a7b8180ff5a0c05df54/users/0cb24d67d6b2449089dc685e3ff2b1eb/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:59.012498 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbb008-750d-45d8-b363-5161dededce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a3606789f34a7b8180ff5a0c05df54, user_id=0cb24d67d6b2449089dc685e3ff2b1eb, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:59.015439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fbb008-750d-45d8-b363-5161dededce4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:59.093360 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1379/2742] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:58 2022] PUT /identity/v3/projects/90a3606789f34a7b8180ff5a0c05df54/users/0cb24d67d6b2449089dc685e3ff2b1eb/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:59.095437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf1b18-f00f-45a1-ac28-9b214551afa3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:59.107069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf1b18-f00f-45a1-ac28-9b214551afa3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:59.167987 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa7052-6612-4aba-a5d5-3d4ca11201b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:59.168630 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa7052-6612-4aba-a5d5-3d4ca11201b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:59.168920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa7052-6612-4aba-a5d5-3d4ca11201b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:59.169231 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa7052-6612-4aba-a5d5-3d4ca11201b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:59.169956 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa7052-6612-4aba-a5d5-3d4ca11201b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:59.171570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa7052-6612-4aba-a5d5-3d4ca11201b8 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:59.182163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1364/2743] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:43:59.186116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbb008-750d-45d8-b363-5161dededce4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:59.196514 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fbb008-750d-45d8-b363-5161dededce4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:59.205373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:59.206266 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:59.206553 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:59.206883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] PATH_INFO: `/v3/projects/90a3606789f34a7b8180ff5a0c05df54/users/0cb24d67d6b2449089dc685e3ff2b1eb/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:59.209116 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a3606789f34a7b8180ff5a0c05df54, user_id=0cb24d67d6b2449089dc685e3ff2b1eb, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:59.212380 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:59.224464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1380/2744] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:43:59 2022] PUT /identity/v3/projects/90a3606789f34a7b8180ff5a0c05df54/users/0cb24d67d6b2449089dc685e3ff2b1eb/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:43:59.236488 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd4572-74cb-4ee5-abf0-1524b0cdfed1 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:59.236858 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd4572-74cb-4ee5-abf0-1524b0cdfed1 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:59.237055 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd4572-74cb-4ee5-abf0-1524b0cdfed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:59.267875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-8fbd4572-74cb-4ee5-abf0-1524b0cdfed1 None None] MFA Rules not processed for user `0cb24d67d6b2449089dc685e3ff2b1eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:43:59.269086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-8fbd4572-74cb-4ee5-abf0-1524b0cdfed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:59.339979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1365/2745] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:43:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:43:59.358302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:59.358302 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:43:59.371101 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:59.420135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1ccd6-6f70-4e64-92d4-ea70f845ce20 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:59.424018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1774e-6d7c-4a6c-955b-bfef08d821f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:59.424673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1774e-6d7c-4a6c-955b-bfef08d821f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:59.424952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1774e-6d7c-4a6c-955b-bfef08d821f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:59.425283 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1774e-6d7c-4a6c-955b-bfef08d821f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:59.435016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1774e-6d7c-4a6c-955b-bfef08d821f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:59.436447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1774e-6d7c-4a6c-955b-bfef08d821f5 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:59.447477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1381/2746] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:43:59.621782 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd4572-74cb-4ee5-abf0-1524b0cdfed1 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:43:59.635370 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd4572-74cb-4ee5-abf0-1524b0cdfed1 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:43:59.638467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdffc7-2231-47ae-aa96-85a24783917d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:43:59.638720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdffc7-2231-47ae-aa96-85a24783917d service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:43:59.639015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdffc7-2231-47ae-aa96-85a24783917d service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:43:59.639349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdffc7-2231-47ae-aa96-85a24783917d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:43:59.695340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:43:59.706393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdffc7-2231-47ae-aa96-85a24783917d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:43:59.708136 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdffc7-2231-47ae-aa96-85a24783917d service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:43:59.720176 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1366/2747] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:43:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:00.274303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1774e-6d7c-4a6c-955b-bfef08d821f5 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:00.343781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1774e-6d7c-4a6c-955b-bfef08d821f5 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:00.350487 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:00.352499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:00.354620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:00.354620 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] PATH_INFO: `/v3/users/9bd5ccdeb434457bac9560aa5eca73e1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:00.355880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd5ccdeb434457bac9560aa5eca73e1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:00.358491 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:00.388381 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143176f-674c-496f-a305-2620b523c60e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:00.388842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143176f-674c-496f-a305-2620b523c60e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:00.388957 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143176f-674c-496f-a305-2620b523c60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:00.419025 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1143176f-674c-496f-a305-2620b523c60e None None] MFA Rules not processed for user `045a8e388a02463aada268462f3f5905`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:00.420299 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1143176f-674c-496f-a305-2620b523c60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:00.512525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1367/2748] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:00 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:00.538190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd5ccdeb434457bac9560aa5eca73e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:00.547286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd5ccdeb434457bac9560aa5eca73e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:00.567643 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd5ccdeb434457bac9560aa5eca73e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:00.568622 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd5ccdeb434457bac9560aa5eca73e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:00.661356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd5ccdeb434457bac9560aa5eca73e1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:00.667898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1382/2749] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:00 2022] DELETE /identity/v3/users/9bd5ccdeb434457bac9560aa5eca73e1 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:00.677828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83784-f868-4162-ae71-e5ffc9e743bb None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:00.677828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83784-f868-4162-ae71-e5ffc9e743bb None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:00.677828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e83784-f868-4162-ae71-e5ffc9e743bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:00.693625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f4e83784-f868-4162-ae71-e5ffc9e743bb None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:00.694892 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f4e83784-f868-4162-ae71-e5ffc9e743bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:00.755965 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1368/2750] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:00.764525 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:00.765984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:00.802722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:00.826942 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9669-8c0d-46c9-9105-4b237139db71 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:00.830737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29ca1-df05-4f5b-8aa1-1a464d3d0eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:00.831389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29ca1-df05-4f5b-8aa1-1a464d3d0eef service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:00.831673 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29ca1-df05-4f5b-8aa1-1a464d3d0eef service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:00.831952 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29ca1-df05-4f5b-8aa1-1a464d3d0eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:00.841865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf29ca1-df05-4f5b-8aa1-1a464d3d0eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:00.843293 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf29ca1-df05-4f5b-8aa1-1a464d3d0eef service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:00.855135 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1383/2751] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:01.308000 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e83784-f868-4162-ae71-e5ffc9e743bb None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:01.309502 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f4e83784-f868-4162-ae71-e5ffc9e743bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:01.322129 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e83784-f868-4162-ae71-e5ffc9e743bb None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:01.327028 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:01.327907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:01.328193 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:01.328474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] PATH_INFO: `/v3/projects/77ce20a4fca84c419e1a561027d784b3` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:01.329710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ce20a4fca84c419e1a561027d784b3)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:01.331632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:01.517356 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ce20a4fca84c419e1a561027d784b3', 'request_id': 'req-ce5bcecd-776d-4473-8385-f5a53d306356'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.524602 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ce20a4fca84c419e1a561027d784b3', 'request_id': 'req-ce5bcecd-776d-4473-8385-f5a53d306356'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.531887 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ce20a4fca84c419e1a561027d784b3', 'request_id': 'req-ce5bcecd-776d-4473-8385-f5a53d306356'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.637656 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:01.642761 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] The token cache is being invalidate because project 77ce20a4fca84c419e1a561027d784b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:01.643058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.643690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.645393 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1369/2752] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:01 2022] DELETE /identity/v3/projects/77ce20a4fca84c419e1a561027d784b3 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:01.650211 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29ca1-df05-4f5b-8aa1-1a464d3d0eef service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:01.660184 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29ca1-df05-4f5b-8aa1-1a464d3d0eef service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:01.719186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:01.720102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:01.720389 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:01.720754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] PATH_INFO: `/v3/users/0cb24d67d6b2449089dc685e3ff2b1eb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:01.721965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb24d67d6b2449089dc685e3ff2b1eb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:01.723862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:01.886579 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb24d67d6b2449089dc685e3ff2b1eb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.894983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb24d67d6b2449089dc685e3ff2b1eb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.911433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb24d67d6b2449089dc685e3ff2b1eb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.912233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb24d67d6b2449089dc685e3ff2b1eb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.980793 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb24d67d6b2449089dc685e3ff2b1eb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:01.987231 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1384/2753] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:01 2022] DELETE /identity/v3/users/0cb24d67d6b2449089dc685e3ff2b1eb => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:02.028056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:02.029698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:02.049084 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bcecd-776d-4473-8385-f5a53d306356 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:02.103015 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:02.103806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:02.104093 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:02.104374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] PATH_INFO: `/v3/users/73481e3bc60148479b2c5b36c2da5fb7` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:02.107050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=73481e3bc60148479b2c5b36c2da5fb7)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:02.109003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:02.337836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73481e3bc60148479b2c5b36c2da5fb7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.351920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73481e3bc60148479b2c5b36c2da5fb7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.352625 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73481e3bc60148479b2c5b36c2da5fb7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.353317 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73481e3bc60148479b2c5b36c2da5fb7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.419163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73481e3bc60148479b2c5b36c2da5fb7'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.432307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1370/2754] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:02 2022] DELETE /identity/v3/users/73481e3bc60148479b2c5b36c2da5fb7 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:02.458684 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:02.460148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:02.473335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710ada4-bb63-4447-8b1f-d3c001e687fc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:02.533134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:02.540574 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:02.541489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:02.541784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:02.542069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] PATH_INFO: `/v3/projects/90a3606789f34a7b8180ff5a0c05df54` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:02.543271 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a3606789f34a7b8180ff5a0c05df54)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:02.545361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:02.654457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a3606789f34a7b8180ff5a0c05df54', 'request_id': 'req-5c0839c4-9b0c-447d-a73c-0778b1e6132d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.660846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a3606789f34a7b8180ff5a0c05df54', 'request_id': 'req-5c0839c4-9b0c-447d-a73c-0778b1e6132d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.666951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a3606789f34a7b8180ff5a0c05df54', 'request_id': 'req-5c0839c4-9b0c-447d-a73c-0778b1e6132d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.794756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:02.796307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:02.813024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] The token cache is being invalidate because project 90a3606789f34a7b8180ff5a0c05df54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:02.813373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.813984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5c0839c4-9b0c-447d-a73c-0778b1e6132d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:02.814710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc7df-463b-4a5a-b23a-394a236dc8dc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:02.815760 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1385/2755] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:02 2022] DELETE /identity/v3/projects/90a3606789f34a7b8180ff5a0c05df54 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:02.820921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07264e19-fd1d-4855-8a36-6a2c7050260e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:02.821161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07264e19-fd1d-4855-8a36-6a2c7050260e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:02.821447 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07264e19-fd1d-4855-8a36-6a2c7050260e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:02.842887 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-07264e19-fd1d-4855-8a36-6a2c7050260e None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:02.844399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-07264e19-fd1d-4855-8a36-6a2c7050260e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:02.883384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:02.884259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:02.884546 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:02.884810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] PATH_INFO: `/v3/projects/92a6848c84c84a84847deda84133204c` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:02.886070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a6848c84c84a84847deda84133204c)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:02.893173 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:02.915429 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1386/2756] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:44:02.922754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee1be-45a4-479e-9e24-a9083427057c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:02.923051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee1be-45a4-479e-9e24-a9083427057c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:02.928941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ee1be-45a4-479e-9e24-a9083427057c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:02.929253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:02.942653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3b7ee1be-45a4-479e-9e24-a9083427057c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:02.943844 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3b7ee1be-45a4-479e-9e24-a9083427057c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:02.953307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1387/2757] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:44:02.957010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ee1be-45a4-479e-9e24-a9083427057c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:02.966495 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ee1be-45a4-479e-9e24-a9083427057c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:02.971147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:02.971840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:02.972077 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:02.972359 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:02.973201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:44:02.973422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:02.975016 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:02.986729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1388/2758] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:44:02.994653 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:03.004595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc3da3-ead7-41a3-94ac-56aab3213e8c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:03.010170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cce231-5a63-4c98-a9e0-a174df4070ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:03.010821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cce231-5a63-4c98-a9e0-a174df4070ca admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.011110 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cce231-5a63-4c98-a9e0-a174df4070ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.011454 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cce231-5a63-4c98-a9e0-a174df4070ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.021066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cce231-5a63-4c98-a9e0-a174df4070ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:03.021066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cce231-5a63-4c98-a9e0-a174df4070ca admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:03.035736 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a6848c84c84a84847deda84133204c', 'request_id': 'req-b565cc3f-e452-45dc-b6b4-06b57b653e2f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.041867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a6848c84c84a84847deda84133204c', 'request_id': 'req-b565cc3f-e452-45dc-b6b4-06b57b653e2f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.047559 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a6848c84c84a84847deda84133204c', 'request_id': 'req-b565cc3f-e452-45dc-b6b4-06b57b653e2f'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.091191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1389/2759] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:44:02 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:44:03.095219 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cce231-5a63-4c98-a9e0-a174df4070ca admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:03.146290 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] The token cache is being invalidate because project 92a6848c84c84a84847deda84133204c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:03.146582 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.147201 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.149566 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1371/2760] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:02 2022] DELETE /identity/v3/projects/92a6848c84c84a84847deda84133204c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:03.153200 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:03.154513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:03.157915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cce231-5a63-4c98-a9e0-a174df4070ca admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:03.163621 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:03.164623 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.164911 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.165274 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.166328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:03.168821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:03.217166 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565cc3f-e452-45dc-b6b4-06b57b653e2f admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:03.221827 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:03.222696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.222979 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.223259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] PATH_INFO: `/v3/users/045a8e388a02463aada268462f3f5905` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.224437 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=045a8e388a02463aada268462f3f5905)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:03.226405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:03.267009 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1390/2761] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:44:03 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:44:03.273061 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:03.275276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:03.287284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92e323-8e65-4e73-8e1a-0a63a25362ed admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:03.291969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442485f-12ff-4e82-b262-1bc2f5e35534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:03.293712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442485f-12ff-4e82-b262-1bc2f5e35534 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.293712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442485f-12ff-4e82-b262-1bc2f5e35534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.293712 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442485f-12ff-4e82-b262-1bc2f5e35534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.294023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442485f-12ff-4e82-b262-1bc2f5e35534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:03.295560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442485f-12ff-4e82-b262-1bc2f5e35534 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:03.305452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1391/2762] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:44:03.309091 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442485f-12ff-4e82-b262-1bc2f5e35534 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:03.318528 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442485f-12ff-4e82-b262-1bc2f5e35534 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:03.323777 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a631eb-6e10-4864-9d30-775c941e653b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:03.324405 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a631eb-6e10-4864-9d30-775c941e653b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.324691 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a631eb-6e10-4864-9d30-775c941e653b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.324965 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a631eb-6e10-4864-9d30-775c941e653b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.325742 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a631eb-6e10-4864-9d30-775c941e653b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:03.327351 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a631eb-6e10-4864-9d30-775c941e653b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:03.338951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1392/2763] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:44:03.340526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:03.342696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a631eb-6e10-4864-9d30-775c941e653b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:03.352722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a631eb-6e10-4864-9d30-775c941e653b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:03.357534 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc054c3d-fd62-4b5a-965d-27194b11d2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:03.358406 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054c3d-fd62-4b5a-965d-27194b11d2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.358692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054c3d-fd62-4b5a-965d-27194b11d2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.358973 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc054c3d-fd62-4b5a-965d-27194b11d2ba admin admin] PATH_INFO: `/v3/projects/4d29a7ef0705457abb2ced74b2875630/users/fe8297ce57e64f2896d9568058c50d04/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.373972 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc054c3d-fd62-4b5a-965d-27194b11d2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d29a7ef0705457abb2ced74b2875630, user_id=fe8297ce57e64f2896d9568058c50d04, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:03.378424 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '045a8e388a02463aada268462f3f5905'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.378564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc054c3d-fd62-4b5a-965d-27194b11d2ba admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:03.388999 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '045a8e388a02463aada268462f3f5905'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.389721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '045a8e388a02463aada268462f3f5905'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.390345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '045a8e388a02463aada268462f3f5905'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.480375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '045a8e388a02463aada268462f3f5905'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.488109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1393/2764] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:03 2022] PUT /identity/v3/projects/4d29a7ef0705457abb2ced74b2875630/users/fe8297ce57e64f2896d9568058c50d04/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:03.494244 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0eb0-facd-45dd-b4cc-87d461c47260 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.496237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0eb0-facd-45dd-b4cc-87d461c47260 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.496359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1372/2765] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:03 2022] DELETE /identity/v3/users/045a8e388a02463aada268462f3f5905 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:03.496455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac0eb0-facd-45dd-b4cc-87d461c47260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.513790 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f7ac0eb0-facd-45dd-b4cc-87d461c47260 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:03.514988 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f7ac0eb0-facd-45dd-b4cc-87d461c47260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:03.577085 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1394/2766] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:03.586710 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:03.588158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:03.620654 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:03.648161 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f52cba-ac3e-4097-ab44-aed434fd4efc admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:03.651914 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb41934-8f10-4a06-b815-47c2ac7d4e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:03.652597 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb41934-8f10-4a06-b815-47c2ac7d4e43 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.652886 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb41934-8f10-4a06-b815-47c2ac7d4e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.653324 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb41934-8f10-4a06-b815-47c2ac7d4e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.663372 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb41934-8f10-4a06-b815-47c2ac7d4e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:03.664836 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb41934-8f10-4a06-b815-47c2ac7d4e43 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:03.675453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1373/2767] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:03.769436 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0eb0-facd-45dd-b4cc-87d461c47260 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:03.770882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f7ac0eb0-facd-45dd-b4cc-87d461c47260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:03.785769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac0eb0-facd-45dd-b4cc-87d461c47260 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:03.791488 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:03.792361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:03.792812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:03.793037 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] PATH_INFO: `/v3/projects/801920a2d9dd4e3f82928cef7f94061b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:03.794295 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] RBAC: Authorizing `identity:delete_project(project_id=801920a2d9dd4e3f82928cef7f94061b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:03.799879 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:03.943213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801920a2d9dd4e3f82928cef7f94061b', 'request_id': 'req-07f27ff2-065d-4390-91fe-78f6f7c3f16e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.949414 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801920a2d9dd4e3f82928cef7f94061b', 'request_id': 'req-07f27ff2-065d-4390-91fe-78f6f7c3f16e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:03.955088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801920a2d9dd4e3f82928cef7f94061b', 'request_id': 'req-07f27ff2-065d-4390-91fe-78f6f7c3f16e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:04.049048 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] The token cache is being invalidate because project 801920a2d9dd4e3f82928cef7f94061b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:04.049637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:04.050052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07f27ff2-065d-4390-91fe-78f6f7c3f16e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:04.052011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1395/2768] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:03 2022] DELETE /identity/v3/projects/801920a2d9dd4e3f82928cef7f94061b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:04.058470 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ada8-9afc-4a5e-bbe4-3f9e13d18900 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.059011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ada8-9afc-4a5e-bbe4-3f9e13d18900 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.059011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ada8-9afc-4a5e-bbe4-3f9e13d18900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.076961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4113ada8-9afc-4a5e-bbe4-3f9e13d18900 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:04.078220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4113ada8-9afc-4a5e-bbe4-3f9e13d18900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:04.143721 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1374/2769] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:04.153595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49342-10f4-435d-ac73-eb89084ad3ba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.153957 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49342-10f4-435d-ac73-eb89084ad3ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.154303 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49342-10f4-435d-ac73-eb89084ad3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.175006 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-19f49342-10f4-435d-ac73-eb89084ad3ba None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:04.176282 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-19f49342-10f4-435d-ac73-eb89084ad3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:04.185055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1396/2770] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:04.189394 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113ada8-9afc-4a5e-bbe4-3f9e13d18900 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:04.199211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113ada8-9afc-4a5e-bbe4-3f9e13d18900 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:04.204130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:04.208181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.208181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.208181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.208181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:44:04.208181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:04.208181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:04.223862 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1375/2771] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 03:44:04.225057 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f49342-10f4-435d-ac73-eb89084ad3ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:04.235318 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:04.249897 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f49342-10f4-435d-ac73-eb89084ad3ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:04.256671 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94dd5c-1944-4c34-acc8-874d5a4f5795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:04.257309 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94dd5c-1944-4c34-acc8-874d5a4f5795 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.257619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94dd5c-1944-4c34-acc8-874d5a4f5795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.259751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94dd5c-1944-4c34-acc8-874d5a4f5795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.259751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94dd5c-1944-4c34-acc8-874d5a4f5795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:04.260539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94dd5c-1944-4c34-acc8-874d5a4f5795 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:04.388719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1397/2772] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:44:04 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:44:04.392576 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:04.403072 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0ff0-abf6-4205-9d3d-174195b01827 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:04.461600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6b741-b0f8-45d1-9a01-971fac19ec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:04.462303 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b741-b0f8-45d1-9a01-971fac19ec69 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.462589 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b741-b0f8-45d1-9a01-971fac19ec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.462867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b741-b0f8-45d1-9a01-971fac19ec69 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.463570 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6b741-b0f8-45d1-9a01-971fac19ec69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:04.465567 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6b741-b0f8-45d1-9a01-971fac19ec69 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:04.564664 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1376/2773] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:44:04 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:44:04.568759 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94dd5c-1944-4c34-acc8-874d5a4f5795 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:04.579685 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94dd5c-1944-4c34-acc8-874d5a4f5795 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:04.584422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fca880-cb81-4f7e-b976-9e27988809e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:04.585093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca880-cb81-4f7e-b976-9e27988809e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.585464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca880-cb81-4f7e-b976-9e27988809e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.585694 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca880-cb81-4f7e-b976-9e27988809e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.587260 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fca880-cb81-4f7e-b976-9e27988809e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:04.588861 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fca880-cb81-4f7e-b976-9e27988809e9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:04.599382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1398/2774] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:04.603407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6b741-b0f8-45d1-9a01-971fac19ec69 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:04.613314 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6b741-b0f8-45d1-9a01-971fac19ec69 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:04.618060 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cf6cd-d290-44f8-ad79-962165e4cd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:04.618713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf6cd-d290-44f8-ad79-962165e4cd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.619050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf6cd-d290-44f8-ad79-962165e4cd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.619331 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cf6cd-d290-44f8-ad79-962165e4cd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.620068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cf6cd-d290-44f8-ad79-962165e4cd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:04.621981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cf6cd-d290-44f8-ad79-962165e4cd92 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:04.631940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1377/2775] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:04.639748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fca880-cb81-4f7e-b976-9e27988809e9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:04.650324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fca880-cb81-4f7e-b976-9e27988809e9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:04.655214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21c40-a75e-4ce4-aade-03fbce505914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:04.656100 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21c40-a75e-4ce4-aade-03fbce505914 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.656455 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21c40-a75e-4ce4-aade-03fbce505914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.656795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21c40-a75e-4ce4-aade-03fbce505914 admin admin] PATH_INFO: `/v3/projects/fdf28f7e29034070a8679e2b0a8f8a01/users/166e4d56be574a939bb724623c45a5d2/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.672314 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d21c40-a75e-4ce4-aade-03fbce505914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf28f7e29034070a8679e2b0a8f8a01, user_id=166e4d56be574a939bb724623c45a5d2, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:04.675046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d21c40-a75e-4ce4-aade-03fbce505914 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:04.797748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1399/2776] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:04 2022] PUT /identity/v3/projects/fdf28f7e29034070a8679e2b0a8f8a01/users/166e4d56be574a939bb724623c45a5d2/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:04.809906 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b237-fd77-42a5-8d2c-a83c94e49842 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.810198 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b237-fd77-42a5-8d2c-a83c94e49842 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.810481 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7b237-fd77-42a5-8d2c-a83c94e49842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.839626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b0a7b237-fd77-42a5-8d2c-a83c94e49842 None None] MFA Rules not processed for user `166e4d56be574a939bb724623c45a5d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:04.840843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b0a7b237-fd77-42a5-8d2c-a83c94e49842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:04.864495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:04.911245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1378/2777] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:04.917493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21c40-a75e-4ce4-aade-03fbce505914 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:04.982750 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375e66-5641-4ee3-b13a-cc895307bf3d None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.983077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375e66-5641-4ee3-b13a-cc895307bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.984101 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d375e66-5641-4ee3-b13a-cc895307bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.984737 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21c40-a75e-4ce4-aade-03fbce505914 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:04.989548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77585b-1dcc-49e1-a869-b093bd95c354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:04.990220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77585b-1dcc-49e1-a869-b093bd95c354 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:04.990504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77585b-1dcc-49e1-a869-b093bd95c354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:04.990783 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77585b-1dcc-49e1-a869-b093bd95c354 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:04.991537 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77585b-1dcc-49e1-a869-b093bd95c354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:04.994114 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77585b-1dcc-49e1-a869-b093bd95c354 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.017286 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-0d375e66-5641-4ee3-b13a-cc895307bf3d None None] MFA Rules not processed for user `fe8297ce57e64f2896d9568058c50d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:05.018501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0d375e66-5641-4ee3-b13a-cc895307bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:05.081387 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1400/2778] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:44:04 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:44:05.084811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1379/2779] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:05.085323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77585b-1dcc-49e1-a869-b093bd95c354 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.089453 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d375e66-5641-4ee3-b13a-cc895307bf3d None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.091094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-0d375e66-5641-4ee3-b13a-cc895307bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:05.109328 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d375e66-5641-4ee3-b13a-cc895307bf3d None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.158609 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77585b-1dcc-49e1-a869-b093bd95c354 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.163319 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce95ff-959d-443b-8054-ee91157dadf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.164075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce95ff-959d-443b-8054-ee91157dadf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.164363 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce95ff-959d-443b-8054-ee91157dadf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.164644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce95ff-959d-443b-8054-ee91157dadf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.165365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce95ff-959d-443b-8054-ee91157dadf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.167280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce95ff-959d-443b-8054-ee91157dadf8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.175660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dc724-461f-48d0-8892-cc9a06156153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.176329 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dc724-461f-48d0-8892-cc9a06156153 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.176618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dc724-461f-48d0-8892-cc9a06156153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.176901 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dc724-461f-48d0-8892-cc9a06156153 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.177633 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dc724-461f-48d0-8892-cc9a06156153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.180092 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98dc724-461f-48d0-8892-cc9a06156153 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.272682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1401/2780] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:44:05 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:44:05.276548 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce95ff-959d-443b-8054-ee91157dadf8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.286565 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce95ff-959d-443b-8054-ee91157dadf8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.287809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1380/2781] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:44:05 2022] POST /identity/v3/projects => generated 371 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:44:05.291785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dc724-461f-48d0-8892-cc9a06156153 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.310045 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:05.343049 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:05.357147 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39bb89-17dc-4de0-8a5b-d235d9863255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.359019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39bb89-17dc-4de0-8a5b-d235d9863255 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.359019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39bb89-17dc-4de0-8a5b-d235d9863255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.359019 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39bb89-17dc-4de0-8a5b-d235d9863255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.359276 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39bb89-17dc-4de0-8a5b-d235d9863255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.360993 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39bb89-17dc-4de0-8a5b-d235d9863255 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.363764 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98dc724-461f-48d0-8892-cc9a06156153 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.369412 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46645-c00a-4f42-924b-7bd170290cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.370620 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46645-c00a-4f42-924b-7bd170290cef admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.371031 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46645-c00a-4f42-924b-7bd170290cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.371399 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46645-c00a-4f42-924b-7bd170290cef admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.372104 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec46645-c00a-4f42-924b-7bd170290cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.372936 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1402/2782] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:44:05.374220 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec46645-c00a-4f42-924b-7bd170290cef admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.377367 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39bb89-17dc-4de0-8a5b-d235d9863255 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.392161 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39bb89-17dc-4de0-8a5b-d235d9863255 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.397144 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731069b-7729-4ca4-a4ed-4684920a7c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.398034 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731069b-7729-4ca4-a4ed-4684920a7c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.398352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731069b-7729-4ca4-a4ed-4684920a7c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.398636 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731069b-7729-4ca4-a4ed-4684920a7c2a admin admin] PATH_INFO: `/v3/projects/a1a4b38117f14bbdb5184af9d14d6a39/users/d963bcc8e1514763a93de36f34812df4/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.415489 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731069b-7729-4ca4-a4ed-4684920a7c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a4b38117f14bbdb5184af9d14d6a39, user_id=d963bcc8e1514763a93de36f34812df4, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.418192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731069b-7729-4ca4-a4ed-4684920a7c2a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.465879 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1381/2783] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:44:05 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:44:05.470793 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46645-c00a-4f42-924b-7bd170290cef admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.480273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46645-c00a-4f42-924b-7bd170290cef admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.486042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27824411-c1e4-4da2-9a7c-f329731aefbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.486786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27824411-c1e4-4da2-9a7c-f329731aefbe admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.486981 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27824411-c1e4-4da2-9a7c-f329731aefbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.487258 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27824411-c1e4-4da2-9a7c-f329731aefbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.487982 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27824411-c1e4-4da2-9a7c-f329731aefbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.489660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27824411-c1e4-4da2-9a7c-f329731aefbe admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.490123 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1403/2784] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:05 2022] PUT /identity/v3/projects/a1a4b38117f14bbdb5184af9d14d6a39/users/d963bcc8e1514763a93de36f34812df4/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:05.493959 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731069b-7729-4ca4-a4ed-4684920a7c2a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.500978 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1382/2785] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:44:05.504988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27824411-c1e4-4da2-9a7c-f329731aefbe admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.515137 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27824411-c1e4-4da2-9a7c-f329731aefbe admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.555167 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731069b-7729-4ca4-a4ed-4684920a7c2a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.559870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a6dda-8f32-4836-b6ff-95120c29981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.560536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a6dda-8f32-4836-b6ff-95120c29981a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.560822 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a6dda-8f32-4836-b6ff-95120c29981a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.561129 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a6dda-8f32-4836-b6ff-95120c29981a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.561856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a6dda-8f32-4836-b6ff-95120c29981a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.563444 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a6dda-8f32-4836-b6ff-95120c29981a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.574066 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1404/2786] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:44:05.574226 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5a249-367a-48e4-a05f-13f25dab1d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.575011 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5a249-367a-48e4-a05f-13f25dab1d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.575300 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5a249-367a-48e4-a05f-13f25dab1d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.575581 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5a249-367a-48e4-a05f-13f25dab1d49 admin admin] PATH_INFO: `/v3/projects/bb70e9dda0234834b4e62731ae35a60e/users/a67985ad85bd44fd9b4f1b2a919e4ca8/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.578096 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a6dda-8f32-4836-b6ff-95120c29981a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.588411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a6dda-8f32-4836-b6ff-95120c29981a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.590697 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5a249-367a-48e4-a05f-13f25dab1d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb70e9dda0234834b4e62731ae35a60e, user_id=a67985ad85bd44fd9b4f1b2a919e4ca8, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.592994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d0dbd-9b99-479a-8e04-1a6f88c3314c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.593449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5a249-367a-48e4-a05f-13f25dab1d49 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.593878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d0dbd-9b99-479a-8e04-1a6f88c3314c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.594196 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d0dbd-9b99-479a-8e04-1a6f88c3314c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.594474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d0dbd-9b99-479a-8e04-1a6f88c3314c admin admin] PATH_INFO: `/v3/projects/a1a4b38117f14bbdb5184af9d14d6a39/users/d963bcc8e1514763a93de36f34812df4/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.596474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d0dbd-9b99-479a-8e04-1a6f88c3314c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a4b38117f14bbdb5184af9d14d6a39, user_id=d963bcc8e1514763a93de36f34812df4, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.599102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0d0dbd-9b99-479a-8e04-1a6f88c3314c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.609356 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1405/2787] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:05 2022] PUT /identity/v3/projects/a1a4b38117f14bbdb5184af9d14d6a39/users/d963bcc8e1514763a93de36f34812df4/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:05.625855 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503deac2-31d3-4752-8e7c-6e93fe17fa39 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.626149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503deac2-31d3-4752-8e7c-6e93fe17fa39 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.626434 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503deac2-31d3-4752-8e7c-6e93fe17fa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.652536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-503deac2-31d3-4752-8e7c-6e93fe17fa39 None None] MFA Rules not processed for user `d963bcc8e1514763a93de36f34812df4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:05.653769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-503deac2-31d3-4752-8e7c-6e93fe17fa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:05.684941 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1383/2788] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:05 2022] PUT /identity/v3/projects/bb70e9dda0234834b4e62731ae35a60e/users/a67985ad85bd44fd9b4f1b2a919e4ca8/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:05.688832 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5a249-367a-48e4-a05f-13f25dab1d49 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.724627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1406/2789] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:44:05.736298 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503deac2-31d3-4752-8e7c-6e93fe17fa39 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.737907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-503deac2-31d3-4752-8e7c-6e93fe17fa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:05.743736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:05.753459 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5a249-367a-48e4-a05f-13f25dab1d49 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.759004 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49b2f6-fd6e-47e4-ba35-4c5e8fa15148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.759734 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49b2f6-fd6e-47e4-ba35-4c5e8fa15148 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.760029 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49b2f6-fd6e-47e4-ba35-4c5e8fa15148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.760333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49b2f6-fd6e-47e4-ba35-4c5e8fa15148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.761058 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49b2f6-fd6e-47e4-ba35-4c5e8fa15148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.762800 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49b2f6-fd6e-47e4-ba35-4c5e8fa15148 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.773217 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1384/2790] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:05.776896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49b2f6-fd6e-47e4-ba35-4c5e8fa15148 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:05.786531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49b2f6-fd6e-47e4-ba35-4c5e8fa15148 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.791208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66e50-7732-4627-aa27-486cc01a6fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.792070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66e50-7732-4627-aa27-486cc01a6fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.792359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66e50-7732-4627-aa27-486cc01a6fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.792638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66e50-7732-4627-aa27-486cc01a6fc7 admin admin] PATH_INFO: `/v3/projects/bb70e9dda0234834b4e62731ae35a60e/users/a67985ad85bd44fd9b4f1b2a919e4ca8/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.794640 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66e50-7732-4627-aa27-486cc01a6fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb70e9dda0234834b4e62731ae35a60e, user_id=a67985ad85bd44fd9b4f1b2a919e4ca8, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.796818 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503deac2-31d3-4752-8e7c-6e93fe17fa39 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:05.797302 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66e50-7732-4627-aa27-486cc01a6fc7 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.808292 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1385/2791] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:05 2022] PUT /identity/v3/projects/bb70e9dda0234834b4e62731ae35a60e/users/a67985ad85bd44fd9b4f1b2a919e4ca8/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:05.850614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a5e34-2d22-49bd-b912-210776635880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:05.851193 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a5e34-2d22-49bd-b912-210776635880 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:05.851500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a5e34-2d22-49bd-b912-210776635880 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:05.851781 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a5e34-2d22-49bd-b912-210776635880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:05.912373 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a5e34-2d22-49bd-b912-210776635880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:05.913878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a5e34-2d22-49bd-b912-210776635880 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:05.922820 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1407/2792] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:44:07.364609 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1e03e-ec37-4dd6-be95-a08869485c33 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:07.365170 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1e03e-ec37-4dd6-be95-a08869485c33 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:07.365291 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1e03e-ec37-4dd6-be95-a08869485c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:07.395062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-dad1e03e-ec37-4dd6-be95-a08869485c33 None None] MFA Rules not processed for user `a67985ad85bd44fd9b4f1b2a919e4ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:07.396345 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-dad1e03e-ec37-4dd6-be95-a08869485c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:07.409374 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:07.467539 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1386/2793] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:07.479847 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a5e34-2d22-49bd-b912-210776635880 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:07.488512 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a5e34-2d22-49bd-b912-210776635880 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:07.542823 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bf31c-3994-469a-8a21-d0584cdb5c7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:07.543438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bf31c-3994-469a-8a21-d0584cdb5c7d service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:07.543721 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bf31c-3994-469a-8a21-d0584cdb5c7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:07.543998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bf31c-3994-469a-8a21-d0584cdb5c7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:07.570923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:07.600493 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bf31c-3994-469a-8a21-d0584cdb5c7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:07.602018 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bf31c-3994-469a-8a21-d0584cdb5c7d service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:07.611256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1408/2794] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:09.163368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1e03e-ec37-4dd6-be95-a08869485c33 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:09.224334 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1e03e-ec37-4dd6-be95-a08869485c33 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:09.229529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:09.230402 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:09.230684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:09.230960 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] PATH_INFO: `/v3/users/166e4d56be574a939bb724623c45a5d2` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:09.232149 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=166e4d56be574a939bb724623c45a5d2)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:09.234118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:09.355546 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bf31c-3994-469a-8a21-d0584cdb5c7d service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:09.364878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bf31c-3994-469a-8a21-d0584cdb5c7d service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:09.369299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbb566-bad4-44b2-ba1b-9607940ff317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:09.370141 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbb566-bad4-44b2-ba1b-9607940ff317 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:09.370690 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbb566-bad4-44b2-ba1b-9607940ff317 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:09.371023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbb566-bad4-44b2-ba1b-9607940ff317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:09.381199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbb566-bad4-44b2-ba1b-9607940ff317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:09.382656 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbb566-bad4-44b2-ba1b-9607940ff317 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:09.392784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1409/2795] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 03:44:09.440586 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166e4d56be574a939bb724623c45a5d2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:09.452553 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166e4d56be574a939bb724623c45a5d2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:09.453268 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166e4d56be574a939bb724623c45a5d2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:09.453898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166e4d56be574a939bb724623c45a5d2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:09.545056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166e4d56be574a939bb724623c45a5d2'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:09.558243 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1387/2796] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:09 2022] DELETE /identity/v3/users/166e4d56be574a939bb724623c45a5d2 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:09.562702 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914498f-8d88-43b7-a8df-5700ca7fc3b4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:09.562998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914498f-8d88-43b7-a8df-5700ca7fc3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:09.563285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914498f-8d88-43b7-a8df-5700ca7fc3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:09.584052 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5914498f-8d88-43b7-a8df-5700ca7fc3b4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:09.585467 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5914498f-8d88-43b7-a8df-5700ca7fc3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:09.645784 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1410/2797] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:09.655962 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:09.657396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:09.689406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:09.714273 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2b86d-f03b-4cb2-b808-8b9c1099d10a admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:09.718039 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4eb02-a55b-4412-9586-d2dc763efa56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:09.718689 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4eb02-a55b-4412-9586-d2dc763efa56 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:09.718968 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4eb02-a55b-4412-9586-d2dc763efa56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:09.719242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4eb02-a55b-4412-9586-d2dc763efa56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:09.729283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4eb02-a55b-4412-9586-d2dc763efa56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:09.730724 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4eb02-a55b-4412-9586-d2dc763efa56 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:09.740162 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1388/2798] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:10.190774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914498f-8d88-43b7-a8df-5700ca7fc3b4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:10.190774 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5914498f-8d88-43b7-a8df-5700ca7fc3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:10.203340 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914498f-8d88-43b7-a8df-5700ca7fc3b4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:10.209335 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:10.210215 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:10.210503 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:10.210779 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] PATH_INFO: `/v3/projects/fdf28f7e29034070a8679e2b0a8f8a01` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:10.213192 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf28f7e29034070a8679e2b0a8f8a01)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:10.215635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:10.328464 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf28f7e29034070a8679e2b0a8f8a01', 'request_id': 'req-8621d9da-9295-4104-8471-07c30749a060'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.334675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf28f7e29034070a8679e2b0a8f8a01', 'request_id': 'req-8621d9da-9295-4104-8471-07c30749a060'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.340878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf28f7e29034070a8679e2b0a8f8a01', 'request_id': 'req-8621d9da-9295-4104-8471-07c30749a060'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.432854 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] The token cache is being invalidate because project fdf28f7e29034070a8679e2b0a8f8a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:10.436157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.436157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.436157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1411/2799] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:10 2022] DELETE /identity/v3/projects/fdf28f7e29034070a8679e2b0a8f8a01 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:10.439183 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4eb02-a55b-4412-9586-d2dc763efa56 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:10.448973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4eb02-a55b-4412-9586-d2dc763efa56 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:10.505646 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:10.506509 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:10.506801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:10.507076 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] PATH_INFO: `/v3/users/d963bcc8e1514763a93de36f34812df4` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:10.508810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d963bcc8e1514763a93de36f34812df4)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:10.510808 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:10.691033 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd963bcc8e1514763a93de36f34812df4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.703380 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd963bcc8e1514763a93de36f34812df4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.704046 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd963bcc8e1514763a93de36f34812df4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.704677 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd963bcc8e1514763a93de36f34812df4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.838172 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-9bd91781-82d8-437e-bd61-3e56413921c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd963bcc8e1514763a93de36f34812df4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:10.851199 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1389/2800] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:10 2022] DELETE /identity/v3/users/d963bcc8e1514763a93de36f34812df4 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:11.256826 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:11.258366 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:11.271726 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621d9da-9295-4104-8471-07c30749a060 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:11.315433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:11.339701 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:11.340573 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:11.340857 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:11.341181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] PATH_INFO: `/v3/projects/a1a4b38117f14bbdb5184af9d14d6a39` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:11.342539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a4b38117f14bbdb5184af9d14d6a39)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:11.344463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:11.477425 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a4b38117f14bbdb5184af9d14d6a39', 'request_id': 'req-fbdfed31-2162-4885-831c-618ed327ab4f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:11.483529 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a4b38117f14bbdb5184af9d14d6a39', 'request_id': 'req-fbdfed31-2162-4885-831c-618ed327ab4f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:11.488941 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a4b38117f14bbdb5184af9d14d6a39', 'request_id': 'req-fbdfed31-2162-4885-831c-618ed327ab4f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:11.597438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] The token cache is being invalidate because project a1a4b38117f14bbdb5184af9d14d6a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:11.597794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:11.598437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fbdfed31-2162-4885-831c-618ed327ab4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:11.600149 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1412/2801] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:11 2022] DELETE /identity/v3/projects/a1a4b38117f14bbdb5184af9d14d6a39 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:11.605003 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca5f56-3562-4767-8122-21ebd2872a5c None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:11.605422 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca5f56-3562-4767-8122-21ebd2872a5c None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:11.605757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca5f56-3562-4767-8122-21ebd2872a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:11.625318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-a2ca5f56-3562-4767-8122-21ebd2872a5c None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:11.626538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a2ca5f56-3562-4767-8122-21ebd2872a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:11.671532 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:11.685799 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1390/2802] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:44:11.693385 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39184a09-a4d4-4158-8584-c736c6ffba97 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:11.693748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39184a09-a4d4-4158-8584-c736c6ffba97 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:11.694038 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39184a09-a4d4-4158-8584-c736c6ffba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:11.712856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-39184a09-a4d4-4158-8584-c736c6ffba97 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:11.714130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-39184a09-a4d4-4158-8584-c736c6ffba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:11.722109 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1413/2803] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:44:11.726423 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca5f56-3562-4767-8122-21ebd2872a5c None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:11.736087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca5f56-3562-4767-8122-21ebd2872a5c None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:11.740637 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:11.741321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:11.741612 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:11.742405 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:11.743131 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:44:11.743444 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:11.745125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:11.757159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1391/2804] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:44:11.762079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39184a09-a4d4-4158-8584-c736c6ffba97 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:11.771484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39184a09-a4d4-4158-8584-c736c6ffba97 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:11.776199 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b330a0-42ba-4e10-9a77-83c74824b721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:11.776871 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b330a0-42ba-4e10-9a77-83c74824b721 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:11.777186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b330a0-42ba-4e10-9a77-83c74824b721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:11.777480 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b330a0-42ba-4e10-9a77-83c74824b721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:11.778188 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b330a0-42ba-4e10-9a77-83c74824b721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:11.780064 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b330a0-42ba-4e10-9a77-83c74824b721 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:11.889662 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1414/2805] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:44:11 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:44:11.893847 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:11.904326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e5c87-7799-4a01-8871-70a22bf3d566 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:11.964408 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df20d8-1463-4f2d-afa3-e9ba2c0786eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:11.965075 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df20d8-1463-4f2d-afa3-e9ba2c0786eb admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:11.965389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df20d8-1463-4f2d-afa3-e9ba2c0786eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:11.965666 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df20d8-1463-4f2d-afa3-e9ba2c0786eb admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:11.966359 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df20d8-1463-4f2d-afa3-e9ba2c0786eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:11.968267 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90df20d8-1463-4f2d-afa3-e9ba2c0786eb admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:12.072940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1392/2806] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:44:11 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 03:44:12.077082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b330a0-42ba-4e10-9a77-83c74824b721 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:12.090349 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b330a0-42ba-4e10-9a77-83c74824b721 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:12.095035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92d009-49ca-4fa8-b46f-2c4c5171242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:12.098166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92d009-49ca-4fa8-b46f-2c4c5171242f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:12.098166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92d009-49ca-4fa8-b46f-2c4c5171242f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:12.098166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92d009-49ca-4fa8-b46f-2c4c5171242f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:12.098166 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92d009-49ca-4fa8-b46f-2c4c5171242f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:12.098614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92d009-49ca-4fa8-b46f-2c4c5171242f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:12.109138 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1415/2807] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:12.113260 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df20d8-1463-4f2d-afa3-e9ba2c0786eb admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:12.123655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df20d8-1463-4f2d-afa3-e9ba2c0786eb admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:12.128320 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7c82d-c69d-45ac-9eef-b5f096dc7988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:12.128963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c82d-c69d-45ac-9eef-b5f096dc7988 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:12.129306 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c82d-c69d-45ac-9eef-b5f096dc7988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:12.129584 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c82d-c69d-45ac-9eef-b5f096dc7988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:12.130326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7c82d-c69d-45ac-9eef-b5f096dc7988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:12.131903 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7c82d-c69d-45ac-9eef-b5f096dc7988 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:12.142739 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1393/2808] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:12.146624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92d009-49ca-4fa8-b46f-2c4c5171242f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:12.157392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92d009-49ca-4fa8-b46f-2c4c5171242f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:12.162084 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:12.162997 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:12.163277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:12.163549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] PATH_INFO: `/v3/projects/4b3abf46ab8e4acf9c28a566153fa6ec/users/4d8304f5dd0c4249b45e2959cc53cbea/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:12.178036 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b3abf46ab8e4acf9c28a566153fa6ec, user_id=4d8304f5dd0c4249b45e2959cc53cbea, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:12.180675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:12.262165 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1416/2809] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:12 2022] PUT /identity/v3/projects/4b3abf46ab8e4acf9c28a566153fa6ec/users/4d8304f5dd0c4249b45e2959cc53cbea/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:12.278384 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b124bc2-6224-4f46-9413-1c7739747489 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:12.278674 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b124bc2-6224-4f46-9413-1c7739747489 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:12.278971 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b124bc2-6224-4f46-9413-1c7739747489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:12.305527 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1b124bc2-6224-4f46-9413-1c7739747489 None None] MFA Rules not processed for user `4d8304f5dd0c4249b45e2959cc53cbea`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:12.306720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1b124bc2-6224-4f46-9413-1c7739747489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:12.369943 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1394/2810] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:12.381535 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:12.383821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:12.438614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c2ba0-a324-46c2-a66b-8109e2bfd313 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:12.442270 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfee5e-1b0f-49cb-8668-e02fcd5c4278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:12.442929 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfee5e-1b0f-49cb-8668-e02fcd5c4278 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:12.443213 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfee5e-1b0f-49cb-8668-e02fcd5c4278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:12.443492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfee5e-1b0f-49cb-8668-e02fcd5c4278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:12.452342 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfee5e-1b0f-49cb-8668-e02fcd5c4278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:12.453802 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfee5e-1b0f-49cb-8668-e02fcd5c4278 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:12.462433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1417/2811] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:22.366495 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:22.458461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:31.793007 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b124bc2-6224-4f46-9413-1c7739747489 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:31.794493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1b124bc2-6224-4f46-9413-1c7739747489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:31.869265 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b124bc2-6224-4f46-9413-1c7739747489 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:31.874094 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:31.874961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:31.875245 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:31.875519 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] PATH_INFO: `/v3/users/fe8297ce57e64f2896d9568058c50d04` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:31.876704 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe8297ce57e64f2896d9568058c50d04)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:31.878657 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:32.044805 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe8297ce57e64f2896d9568058c50d04'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.056973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe8297ce57e64f2896d9568058c50d04'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.057670 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe8297ce57e64f2896d9568058c50d04'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.058296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe8297ce57e64f2896d9568058c50d04'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.118158 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe8297ce57e64f2896d9568058c50d04'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.132100 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1395/2812] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:31 2022] DELETE /identity/v3/users/fe8297ce57e64f2896d9568058c50d04 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:32.410051 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfee5e-1b0f-49cb-8668-e02fcd5c4278 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:32.426080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfee5e-1b0f-49cb-8668-e02fcd5c4278 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:32.484382 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:32.485279 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:32.485562 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:32.485840 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] PATH_INFO: `/v3/projects/4d29a7ef0705457abb2ced74b2875630` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:32.487000 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d29a7ef0705457abb2ced74b2875630)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:32.488905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:32.591055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d29a7ef0705457abb2ced74b2875630', 'request_id': 'req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.597186 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d29a7ef0705457abb2ced74b2875630', 'request_id': 'req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.603146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d29a7ef0705457abb2ced74b2875630', 'request_id': 'req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.700715 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] The token cache is being invalidate because project 4d29a7ef0705457abb2ced74b2875630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:32.700963 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.701657 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.703214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1418/2813] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:32 2022] DELETE /identity/v3/projects/4d29a7ef0705457abb2ced74b2875630 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:32.707169 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:32.708579 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:32.722525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb48a9-a0bf-4a74-a368-3a3596562288 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:32.779333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:32.780293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:32.780577 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:32.780854 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] PATH_INFO: `/v3/users/a67985ad85bd44fd9b4f1b2a919e4ca8` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:32.782091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67985ad85bd44fd9b4f1b2a919e4ca8)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:32.784012 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:32.805501 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:32.978436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67985ad85bd44fd9b4f1b2a919e4ca8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.990091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67985ad85bd44fd9b4f1b2a919e4ca8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.990840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67985ad85bd44fd9b4f1b2a919e4ca8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:32.991406 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67985ad85bd44fd9b4f1b2a919e4ca8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:33.070733 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-ff53fbca-ad2b-4b9d-a58e-b40b12bdcac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67985ad85bd44fd9b4f1b2a919e4ca8'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:33.083368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1396/2814] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:32 2022] DELETE /identity/v3/users/a67985ad85bd44fd9b4f1b2a919e4ca8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:33.323795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:33.325376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:33.339134 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51a721-a903-4d8f-a7c3-3344ee82cbb1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:33.402696 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:33.403463 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:33.403741 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:33.404015 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] PATH_INFO: `/v3/projects/bb70e9dda0234834b4e62731ae35a60e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:33.405237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb70e9dda0234834b4e62731ae35a60e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:33.407164 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:33.500980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:33.512323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb70e9dda0234834b4e62731ae35a60e', 'request_id': 'req-dc049238-5a07-4160-b0b0-e038f19d1c3c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:33.518875 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb70e9dda0234834b4e62731ae35a60e', 'request_id': 'req-dc049238-5a07-4160-b0b0-e038f19d1c3c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:33.524264 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb70e9dda0234834b4e62731ae35a60e', 'request_id': 'req-dc049238-5a07-4160-b0b0-e038f19d1c3c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:33.602610 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] The token cache is being invalidate because project bb70e9dda0234834b4e62731ae35a60e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:33.602903 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:33.603554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-dc049238-5a07-4160-b0b0-e038f19d1c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:33.605239 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1419/2815] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:33 2022] DELETE /identity/v3/projects/bb70e9dda0234834b4e62731ae35a60e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:33.611667 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558858f-4dc9-43c0-87ea-e0d301d167b8 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:33.611961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558858f-4dc9-43c0-87ea-e0d301d167b8 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:33.612242 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558858f-4dc9-43c0-87ea-e0d301d167b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:33.631535 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1558858f-4dc9-43c0-87ea-e0d301d167b8 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:33.632788 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1558858f-4dc9-43c0-87ea-e0d301d167b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:33.693616 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1397/2816] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:33.702065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fac591-4e55-43a1-850b-ee6dfa11f3a4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:33.702358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fac591-4e55-43a1-850b-ee6dfa11f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:33.702649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fac591-4e55-43a1-850b-ee6dfa11f3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:33.721554 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-99fac591-4e55-43a1-850b-ee6dfa11f3a4 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:33.722933 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-99fac591-4e55-43a1-850b-ee6dfa11f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:33.732069 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1420/2817] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:33.736627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558858f-4dc9-43c0-87ea-e0d301d167b8 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:33.746556 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558858f-4dc9-43c0-87ea-e0d301d167b8 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:33.753296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:33.753296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:33.753296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:33.753296 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:33.753789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:44:33.753789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:33.755318 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:33.767375 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1398/2818] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:44:33.771587 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fac591-4e55-43a1-850b-ee6dfa11f3a4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:33.780930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fac591-4e55-43a1-850b-ee6dfa11f3a4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:33.785700 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd786500-3bf7-44ea-bef9-6cf685138cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:33.786344 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd786500-3bf7-44ea-bef9-6cf685138cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:33.786649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd786500-3bf7-44ea-bef9-6cf685138cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:33.786923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd786500-3bf7-44ea-bef9-6cf685138cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:33.787614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd786500-3bf7-44ea-bef9-6cf685138cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:33.790079 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd786500-3bf7-44ea-bef9-6cf685138cbe admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:33.860013 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1421/2819] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:44:33 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:44:33.864225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:33.874594 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fbbab-f0d6-4fd8-b10d-c0d1428c6237 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:33.932762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bf602-dbad-400b-8228-363f37970e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:33.933480 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bf602-dbad-400b-8228-363f37970e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:33.933762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bf602-dbad-400b-8228-363f37970e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:33.934040 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bf602-dbad-400b-8228-363f37970e97 admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:33.934737 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bf602-dbad-400b-8228-363f37970e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:33.936653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bf602-dbad-400b-8228-363f37970e97 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:34.017218 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1399/2820] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:44:33 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:44:34.019060 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd786500-3bf7-44ea-bef9-6cf685138cbe admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:34.029827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd786500-3bf7-44ea-bef9-6cf685138cbe admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:34.034624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7905b5-74ce-4d13-8723-d5e1d6bc3ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:34.035286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7905b5-74ce-4d13-8723-d5e1d6bc3ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:34.035569 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7905b5-74ce-4d13-8723-d5e1d6bc3ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:34.036740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7905b5-74ce-4d13-8723-d5e1d6bc3ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:34.036740 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7905b5-74ce-4d13-8723-d5e1d6bc3ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:34.038294 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7905b5-74ce-4d13-8723-d5e1d6bc3ca6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:34.048923 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1422/2821] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:34.052755 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bf602-dbad-400b-8228-363f37970e97 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:34.062565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bf602-dbad-400b-8228-363f37970e97 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:34.067212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9e3e4-252f-4316-ab0d-9b48e308ad10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:34.067861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9e3e4-252f-4316-ab0d-9b48e308ad10 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:34.068148 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9e3e4-252f-4316-ab0d-9b48e308ad10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:34.068427 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9e3e4-252f-4316-ab0d-9b48e308ad10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:34.069253 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9e3e4-252f-4316-ab0d-9b48e308ad10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:34.071791 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9e3e4-252f-4316-ab0d-9b48e308ad10 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:34.081575 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1400/2822] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:34.086299 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7905b5-74ce-4d13-8723-d5e1d6bc3ca6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:34.095930 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7905b5-74ce-4d13-8723-d5e1d6bc3ca6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:34.100479 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:34.101395 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:34.101675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:34.101947 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] PATH_INFO: `/v3/projects/d1d9f51a3cc44062af827a498842ee80/users/303285e036734e5cbd491920ae7473d9/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:34.115865 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d9f51a3cc44062af827a498842ee80, user_id=303285e036734e5cbd491920ae7473d9, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:34.118984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:34.185994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1423/2823] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:34 2022] PUT /identity/v3/projects/d1d9f51a3cc44062af827a498842ee80/users/303285e036734e5cbd491920ae7473d9/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:34.195462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a56657-9140-4732-af18-6d271a59ec37 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:34.195762 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a56657-9140-4732-af18-6d271a59ec37 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:34.196050 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a56657-9140-4732-af18-6d271a59ec37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:34.203525 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:34.214396 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-19a56657-9140-4732-af18-6d271a59ec37 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:34.215600 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-19a56657-9140-4732-af18-6d271a59ec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:34.273768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1401/2824] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:34.283171 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:34.288900 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:34.341744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aac14e-d2f5-4141-a25a-69cddd0c61fa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:34.345399 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40baf26-e5d7-49cf-9c3e-eb2fd2c67f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:34.346035 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40baf26-e5d7-49cf-9c3e-eb2fd2c67f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:34.346320 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40baf26-e5d7-49cf-9c3e-eb2fd2c67f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:34.346600 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40baf26-e5d7-49cf-9c3e-eb2fd2c67f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:34.355422 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40baf26-e5d7-49cf-9c3e-eb2fd2c67f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:34.356850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40baf26-e5d7-49cf-9c3e-eb2fd2c67f87 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:34.366078 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1424/2825] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:34.608709 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a56657-9140-4732-af18-6d271a59ec37 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:34.610144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-19a56657-9140-4732-af18-6d271a59ec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:34.622618 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a56657-9140-4732-af18-6d271a59ec37 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:34.627181 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:34.628042 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:34.628326 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:34.628604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] PATH_INFO: `/v3/users/4d8304f5dd0c4249b45e2959cc53cbea` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:34.629787 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8304f5dd0c4249b45e2959cc53cbea)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:34.631701 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:34.828684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8304f5dd0c4249b45e2959cc53cbea'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:34.842128 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8304f5dd0c4249b45e2959cc53cbea'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:34.842861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8304f5dd0c4249b45e2959cc53cbea'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:34.843568 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8304f5dd0c4249b45e2959cc53cbea'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:34.906829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8304f5dd0c4249b45e2959cc53cbea'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:34.927583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1402/2826] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:44:34 2022] DELETE /identity/v3/users/4d8304f5dd0c4249b45e2959cc53cbea => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:34.932352 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff5155-8024-4a34-a8b8-fd885fcf1ba9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:34.932846 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff5155-8024-4a34-a8b8-fd885fcf1ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:34.932935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff5155-8024-4a34-a8b8-fd885fcf1ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:34.951457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7cff5155-8024-4a34-a8b8-fd885fcf1ba9 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:34.955492 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7cff5155-8024-4a34-a8b8-fd885fcf1ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:35.005842 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:35.014980 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1425/2827] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:35.024961 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:35.027120 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:35.090087 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449752a3-3b4c-47bb-8939-5e3e85f4fce3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:35.093653 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd029dab-43c9-4819-a687-9c6f89bed0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:35.094376 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029dab-43c9-4819-a687-9c6f89bed0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:35.094662 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029dab-43c9-4819-a687-9c6f89bed0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:35.094940 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029dab-43c9-4819-a687-9c6f89bed0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:35.104230 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd029dab-43c9-4819-a687-9c6f89bed0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:35.105722 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd029dab-43c9-4819-a687-9c6f89bed0c0 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:35.115333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1403/2828] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:35.783249 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff5155-8024-4a34-a8b8-fd885fcf1ba9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:35.784720 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7cff5155-8024-4a34-a8b8-fd885fcf1ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:35.797220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff5155-8024-4a34-a8b8-fd885fcf1ba9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:35.801832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:35.802699 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:35.802983 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:35.803261 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] PATH_INFO: `/v3/projects/4b3abf46ab8e4acf9c28a566153fa6ec` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:35.804682 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b3abf46ab8e4acf9c28a566153fa6ec)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:35.806619 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:35.928151 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b3abf46ab8e4acf9c28a566153fa6ec', 'request_id': 'req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:35.935103 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b3abf46ab8e4acf9c28a566153fa6ec', 'request_id': 'req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:35.941237 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b3abf46ab8e4acf9c28a566153fa6ec', 'request_id': 'req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:36.032226 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] The token cache is being invalidate because project 4b3abf46ab8e4acf9c28a566153fa6ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:44:36.032675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:36.033627 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3cb91c31-3d1b-43a9-a61a-e7c40c526e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:44:36.038883 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1426/2829] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:44:35 2022] DELETE /identity/v3/projects/4b3abf46ab8e4acf9c28a566153fa6ec => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:36.040631 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddf24d-fa07-410a-b56c-8487d96813d9 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.041473 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddf24d-fa07-410a-b56c-8487d96813d9 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.042389 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddf24d-fa07-410a-b56c-8487d96813d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.062583 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f7ddf24d-fa07-410a-b56c-8487d96813d9 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:36.063807 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f7ddf24d-fa07-410a-b56c-8487d96813d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:36.078655 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:36.126626 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1404/2830] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:36.136285 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53642898-da65-4902-9494-4ab61e5d2896 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.142614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53642898-da65-4902-9494-4ab61e5d2896 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.142614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53642898-da65-4902-9494-4ab61e5d2896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.171681 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-53642898-da65-4902-9494-4ab61e5d2896 None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:36.172994 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-53642898-da65-4902-9494-4ab61e5d2896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:36.182210 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1427/2831] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:36.186945 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddf24d-fa07-410a-b56c-8487d96813d9 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:36.197192 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddf24d-fa07-410a-b56c-8487d96813d9 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:36.202351 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:36.203018 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.205873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.205873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.205873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80355) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 03:44:36.205873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:36.206285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:36.234227 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1405/2832] 10.222.0.41 () {58 vars in 1152 bytes} [Sat May 28 03:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 03:44:36.235769 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53642898-da65-4902-9494-4ab61e5d2896 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:36.246046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53642898-da65-4902-9494-4ab61e5d2896 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:36.250745 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61604a37-07ef-4b13-a10c-3916b9f31c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:36.251461 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61604a37-07ef-4b13-a10c-3916b9f31c52 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.251853 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61604a37-07ef-4b13-a10c-3916b9f31c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.252159 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61604a37-07ef-4b13-a10c-3916b9f31c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.252925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61604a37-07ef-4b13-a10c-3916b9f31c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:36.254974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61604a37-07ef-4b13-a10c-3916b9f31c52 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:36.379307 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1428/2833] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:44:36 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:44:36.383912 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:36.396390 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb892a-472f-4549-aca1-8ba3a47afa93 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:36.460959 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e09e2-1db6-4132-9138-d9639a2815fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:36.462387 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e09e2-1db6-4132-9138-d9639a2815fa admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.462663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e09e2-1db6-4132-9138-d9639a2815fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.463144 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e09e2-1db6-4132-9138-d9639a2815fa admin admin] PATH_INFO: `/v3/users` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.463693 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e09e2-1db6-4132-9138-d9639a2815fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:36.465801 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e09e2-1db6-4132-9138-d9639a2815fa admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:36.572445 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1406/2834] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:44:36 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:44:36.580870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61604a37-07ef-4b13-a10c-3916b9f31c52 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:36.591907 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61604a37-07ef-4b13-a10c-3916b9f31c52 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:36.596580 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3198f1-9d4d-4f46-b14b-396c1d3e5fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:36.599376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3198f1-9d4d-4f46-b14b-396c1d3e5fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.599376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3198f1-9d4d-4f46-b14b-396c1d3e5fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.599376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3198f1-9d4d-4f46-b14b-396c1d3e5fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.599376 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3198f1-9d4d-4f46-b14b-396c1d3e5fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:36.600170 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3198f1-9d4d-4f46-b14b-396c1d3e5fa9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:36.613432 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1429/2835] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:36.621333 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e09e2-1db6-4132-9138-d9639a2815fa admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:36.631893 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e09e2-1db6-4132-9138-d9639a2815fa admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:36.636834 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2e9f64-10f3-4cbd-a88d-5df6ecbb637e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:36.637552 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e9f64-10f3-4cbd-a88d-5df6ecbb637e admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.637845 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e9f64-10f3-4cbd-a88d-5df6ecbb637e admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.638123 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2e9f64-10f3-4cbd-a88d-5df6ecbb637e admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.638851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e9f64-10f3-4cbd-a88d-5df6ecbb637e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:36.640467 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2e9f64-10f3-4cbd-a88d-5df6ecbb637e admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:36.658864 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1407/2836] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:36.662986 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3198f1-9d4d-4f46-b14b-396c1d3e5fa9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:36.674457 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3198f1-9d4d-4f46-b14b-396c1d3e5fa9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:36.679831 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:36.681754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.681754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.681754 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] PATH_INFO: `/v3/projects/282a34892e974900926a37a76690258c/users/20b48a7b54a84c569a1ba39148dc5e70/roles/8e1fb3d93ff549a2899c16dd1130e587` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.705055 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=282a34892e974900926a37a76690258c, user_id=20b48a7b54a84c569a1ba39148dc5e70, role_id=8e1fb3d93ff549a2899c16dd1130e587)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:36.707589 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:36.804388 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1430/2837] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:36 2022] PUT /identity/v3/projects/282a34892e974900926a37a76690258c/users/20b48a7b54a84c569a1ba39148dc5e70/roles/8e1fb3d93ff549a2899c16dd1130e587 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:36.816638 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84488cdc-8fbf-4221-9a6d-98c1e78eecb3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.816931 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84488cdc-8fbf-4221-9a6d-98c1e78eecb3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.817249 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84488cdc-8fbf-4221-9a6d-98c1e78eecb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:36.846604 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-84488cdc-8fbf-4221-9a6d-98c1e78eecb3 None None] MFA Rules not processed for user `20b48a7b54a84c569a1ba39148dc5e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:36.847811 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-84488cdc-8fbf-4221-9a6d-98c1e78eecb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:36.916757 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1408/2838] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:36.927584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:36.929334 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:36.987722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4cce-fabb-4880-8dfb-3572538e0133 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:36.992233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cf63f-db3c-42ec-8797-baf2729f1fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:36.993076 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cf63f-db3c-42ec-8797-baf2729f1fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:36.993564 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cf63f-db3c-42ec-8797-baf2729f1fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:36.993937 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cf63f-db3c-42ec-8797-baf2729f1fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:37.003649 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cf63f-db3c-42ec-8797-baf2729f1fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:37.005296 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52cf63f-db3c-42ec-8797-baf2729f1fd0 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:37.017496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1431/2839] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:38.952802 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec81c9-c929-4757-a42b-5d82f1e9247b None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:38.953341 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec81c9-c929-4757-a42b-5d82f1e9247b None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:38.953436 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec81c9-c929-4757-a42b-5d82f1e9247b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:38.972768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-f3ec81c9-c929-4757-a42b-5d82f1e9247b None None] MFA Rules not processed for user `20b48a7b54a84c569a1ba39148dc5e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:38.974497 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f3ec81c9-c929-4757-a42b-5d82f1e9247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:38.983368 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1409/2840] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:40.077348 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78779e2-685d-4cc3-b378-c0eaebccda18 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:40.078273 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78779e2-685d-4cc3-b378-c0eaebccda18 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:40.078536 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78779e2-685d-4cc3-b378-c0eaebccda18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:40.107549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f78779e2-685d-4cc3-b378-c0eaebccda18 None None] MFA Rules not processed for user `303285e036734e5cbd491920ae7473d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:40.108775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f78779e2-685d-4cc3-b378-c0eaebccda18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:40.121197 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:40.211507 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1432/2841] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:44:40.215632 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec81c9-c929-4757-a42b-5d82f1e9247b None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:40.217054 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-f3ec81c9-c929-4757-a42b-5d82f1e9247b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:40.228905 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:40.313288 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec81c9-c929-4757-a42b-5d82f1e9247b None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:40.326810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9efad-7ac6-41ab-b4fd-76ab9c55fe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:40.327524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9efad-7ac6-41ab-b4fd-76ab9c55fe92 admin admin] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:40.327810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9efad-7ac6-41ab-b4fd-76ab9c55fe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:40.328086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9efad-7ac6-41ab-b4fd-76ab9c55fe92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:40.328785 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9efad-7ac6-41ab-b4fd-76ab9c55fe92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:40.330756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9efad-7ac6-41ab-b4fd-76ab9c55fe92 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:40.455890 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1410/2842] 10.222.0.41 () {60 vars in 1152 bytes} [Sat May 28 03:44:40 2022] POST /identity/v3/projects => generated 373 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:44:40.459430 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78779e2-685d-4cc3-b378-c0eaebccda18 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:40.470894 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78779e2-685d-4cc3-b378-c0eaebccda18 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:40.542277 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f2fcd-d1b9-4041-b225-f1322cef7290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:40.543065 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f2fcd-d1b9-4041-b225-f1322cef7290 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:40.543361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f2fcd-d1b9-4041-b225-f1322cef7290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:40.543647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f2fcd-d1b9-4041-b225-f1322cef7290 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:40.544401 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f2fcd-d1b9-4041-b225-f1322cef7290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:40.546427 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f2fcd-d1b9-4041-b225-f1322cef7290 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:40.667584 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1433/2843] 10.222.0.41 () {60 vars in 1143 bytes} [Sat May 28 03:44:40 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 03:44:40.671936 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9efad-7ac6-41ab-b4fd-76ab9c55fe92 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:40.682877 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9efad-7ac6-41ab-b4fd-76ab9c55fe92 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:40.687528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848c658-245f-427c-9e08-b307a2be4eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:40.690768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848c658-245f-427c-9e08-b307a2be4eca admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:40.690768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848c658-245f-427c-9e08-b307a2be4eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:40.690768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848c658-245f-427c-9e08-b307a2be4eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:40.690768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848c658-245f-427c-9e08-b307a2be4eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:40.691482 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848c658-245f-427c-9e08-b307a2be4eca admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:40.703839 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1411/2844] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 03:44:40.711143 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f2fcd-d1b9-4041-b225-f1322cef7290 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:40.721594 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f2fcd-d1b9-4041-b225-f1322cef7290 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:40.726905 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17f10f-ecef-4361-8b23-13827e68a417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:40.731080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17f10f-ecef-4361-8b23-13827e68a417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:40.731080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17f10f-ecef-4361-8b23-13827e68a417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:40.731080 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17f10f-ecef-4361-8b23-13827e68a417 admin admin] PATH_INFO: `/v3/projects/fad1092444f04842bf6eee17a36bf043/users/286dcf1a1a1c4a2ea8bc8a8d79c4d914/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:40.744466 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17f10f-ecef-4361-8b23-13827e68a417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad1092444f04842bf6eee17a36bf043, user_id=286dcf1a1a1c4a2ea8bc8a8d79c4d914, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:40.747494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17f10f-ecef-4361-8b23-13827e68a417 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:40.863513 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1434/2845] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:40 2022] PUT /identity/v3/projects/fad1092444f04842bf6eee17a36bf043/users/286dcf1a1a1c4a2ea8bc8a8d79c4d914/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:44:40.865404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848c658-245f-427c-9e08-b307a2be4eca admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:40.878013 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848c658-245f-427c-9e08-b307a2be4eca admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:40.938851 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29099242-36aa-4534-954a-ff0e44680e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:40.939538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099242-36aa-4534-954a-ff0e44680e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:40.939828 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099242-36aa-4534-954a-ff0e44680e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:40.940112 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29099242-36aa-4534-954a-ff0e44680e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:40.940842 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29099242-36aa-4534-954a-ff0e44680e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:40.942671 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29099242-36aa-4534-954a-ff0e44680e03 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:40.954165 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1412/2846] 10.222.0.41 () {58 vars in 1121 bytes} [Sat May 28 03:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 03:44:40.958154 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17f10f-ecef-4361-8b23-13827e68a417 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:40.968654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17f10f-ecef-4361-8b23-13827e68a417 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:40.973527 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb403dcc-bd3e-42e2-b29e-b817c3e7d394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:40.974402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403dcc-bd3e-42e2-b29e-b817c3e7d394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:40.974688 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403dcc-bd3e-42e2-b29e-b817c3e7d394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:40.974968 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403dcc-bd3e-42e2-b29e-b817c3e7d394 admin admin] PATH_INFO: `/v3/projects/fad1092444f04842bf6eee17a36bf043/users/286dcf1a1a1c4a2ea8bc8a8d79c4d914/roles/bb0756f8af694633b9743a958eb46e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:40.977027 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb403dcc-bd3e-42e2-b29e-b817c3e7d394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad1092444f04842bf6eee17a36bf043, user_id=286dcf1a1a1c4a2ea8bc8a8d79c4d914, role_id=bb0756f8af694633b9743a958eb46e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:40.979795 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb403dcc-bd3e-42e2-b29e-b817c3e7d394 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:40.993146 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1435/2847] 10.222.0.41 () {60 vars in 1482 bytes} [Sat May 28 03:44:40 2022] PUT /identity/v3/projects/fad1092444f04842bf6eee17a36bf043/users/286dcf1a1a1c4a2ea8bc8a8d79c4d914/roles/bb0756f8af694633b9743a958eb46e7a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:44:46.023700 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed31a7-1fc4-488e-9a42-8dc5ae5139b3 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:46.025347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed31a7-1fc4-488e-9a42-8dc5ae5139b3 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:46.025720 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed31a7-1fc4-488e-9a42-8dc5ae5139b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:46.061414 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-2bed31a7-1fc4-488e-9a42-8dc5ae5139b3 None None] MFA Rules not processed for user `2b515cebf18643c38f1cfc2e3fad1db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:46.062792 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2bed31a7-1fc4-488e-9a42-8dc5ae5139b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:46.173554 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1413/2848] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:50.952832 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455c8e9-5576-4576-a33f-a15f09cc429d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:50.955707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455c8e9-5576-4576-a33f-a15f09cc429d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:50.955707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2455c8e9-5576-4576-a33f-a15f09cc429d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:51.015762 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2455c8e9-5576-4576-a33f-a15f09cc429d None None] MFA Rules not processed for user `286dcf1a1a1c4a2ea8bc8a8d79c4d914`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:51.017281 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2455c8e9-5576-4576-a33f-a15f09cc429d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:51.093265 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:51.118201 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1436/2849] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:44:51.135233 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed31a7-1fc4-488e-9a42-8dc5ae5139b3 None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:51.136009 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-2bed31a7-1fc4-488e-9a42-8dc5ae5139b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:51.173355 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:44:51.224066 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed31a7-1fc4-488e-9a42-8dc5ae5139b3 None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:51.230503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4a996-4c4c-4ef9-96c0-73d6804fac51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:51.231167 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4a996-4c4c-4ef9-96c0-73d6804fac51 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:51.231460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4a996-4c4c-4ef9-96c0-73d6804fac51 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:51.231743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4a996-4c4c-4ef9-96c0-73d6804fac51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:51.295212 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4a996-4c4c-4ef9-96c0-73d6804fac51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:51.296873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4a996-4c4c-4ef9-96c0-73d6804fac51 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:51.308986 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1414/2850] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:44:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:44:54.934642 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268ae86-7af4-4121-9251-2155d7748a9f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:54.935225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268ae86-7af4-4121-9251-2155d7748a9f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:54.935225 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268ae86-7af4-4121-9251-2155d7748a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:54.975550 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d268ae86-7af4-4121-9251-2155d7748a9f None None] MFA Rules not processed for user `303285e036734e5cbd491920ae7473d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:54.978156 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d268ae86-7af4-4121-9251-2155d7748a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:55.022437 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1437/2851] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:44:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:56.281861 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db2041-d583-4aac-bf6c-81836be12f94 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:56.282419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db2041-d583-4aac-bf6c-81836be12f94 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:56.282419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db2041-d583-4aac-bf6c-81836be12f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:56.301391 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-49db2041-d583-4aac-bf6c-81836be12f94 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:44:56.302645 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-49db2041-d583-4aac-bf6c-81836be12f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:56.413056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1415/2852] 10.222.0.41 () {58 vars in 967 bytes} [Sat May 28 03:44:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:44:56.490083 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268ae86-7af4-4121-9251-2155d7748a9f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:44:56.492635 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d268ae86-7af4-4121-9251-2155d7748a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:44:56.632500 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268ae86-7af4-4121-9251-2155d7748a9f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:44:56.647560 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60e545-76d0-4eb2-bae6-acc61e94b78c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:44:56.648652 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60e545-76d0-4eb2-bae6-acc61e94b78c service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:44:56.649256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60e545-76d0-4eb2-bae6-acc61e94b78c service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:44:56.649539 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60e545-76d0-4eb2-bae6-acc61e94b78c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:44:56.659324 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60e545-76d0-4eb2-bae6-acc61e94b78c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:44:56.661058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60e545-76d0-4eb2-bae6-acc61e94b78c service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:44:56.671898 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1438/2853] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:44:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:45:02.685565 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e992fb9-12ce-4673-b2b4-2226db815074 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:02.686088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e992fb9-12ce-4673-b2b4-2226db815074 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:02.686194 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e992fb9-12ce-4673-b2b4-2226db815074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:02.709756 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-4e992fb9-12ce-4673-b2b4-2226db815074 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:45:02.712089 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4e992fb9-12ce-4673-b2b4-2226db815074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:02.720379 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1416/2854] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:45:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:45:02.735770 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60e545-76d0-4eb2-bae6-acc61e94b78c service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:02.752675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60e545-76d0-4eb2-bae6-acc61e94b78c service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:02.758364 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44157c44-ee41-48b5-a0e8-4d37d2d52d44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:02.759011 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44157c44-ee41-48b5-a0e8-4d37d2d52d44 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:02.759297 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44157c44-ee41-48b5-a0e8-4d37d2d52d44 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:02.759582 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44157c44-ee41-48b5-a0e8-4d37d2d52d44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:02.769058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44157c44-ee41-48b5-a0e8-4d37d2d52d44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:02.770677 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44157c44-ee41-48b5-a0e8-4d37d2d52d44 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:02.785328 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1439/2855] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:45:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:45:03.939922 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215d9f-0f7d-4d7d-934b-4e11d21b1c95 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:03.940460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215d9f-0f7d-4d7d-934b-4e11d21b1c95 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:03.940460 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215d9f-0f7d-4d7d-934b-4e11d21b1c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:03.964171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-1b215d9f-0f7d-4d7d-934b-4e11d21b1c95 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:45:03.964171 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1b215d9f-0f7d-4d7d-934b-4e11d21b1c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:03.970778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1417/2856] 10.222.0.41 () {58 vars in 967 bytes} [Sat May 28 03:45:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 03:45:03.990863 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44157c44-ee41-48b5-a0e8-4d37d2d52d44 service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:04.005386 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44157c44-ee41-48b5-a0e8-4d37d2d52d44 service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:04.012801 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41eee6-1d10-481c-bb52-af1facec496c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:04.013872 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41eee6-1d10-481c-bb52-af1facec496c service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:04.014598 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41eee6-1d10-481c-bb52-af1facec496c service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:04.015095 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41eee6-1d10-481c-bb52-af1facec496c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:04.026137 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41eee6-1d10-481c-bb52-af1facec496c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:04.028396 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41eee6-1d10-481c-bb52-af1facec496c service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:04.047799 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1440/2857] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:45:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:45:04.600840 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d7357-565e-4764-9d7b-31f984a10a11 None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:04.601698 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d7357-565e-4764-9d7b-31f984a10a11 None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:04.602068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d7357-565e-4764-9d7b-31f984a10a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:04.621816 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-b72d7357-565e-4764-9d7b-31f984a10a11 None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:45:04.623048 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-b72d7357-565e-4764-9d7b-31f984a10a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:04.630804 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1418/2858] 10.222.0.41 () {58 vars in 1009 bytes} [Sat May 28 03:45:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:45:04.646637 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41eee6-1d10-481c-bb52-af1facec496c service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:04.658836 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41eee6-1d10-481c-bb52-af1facec496c service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:04.672102 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730e79-d1f7-4145-96ae-0c1ea52a98de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:04.673185 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730e79-d1f7-4145-96ae-0c1ea52a98de service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:04.673856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730e79-d1f7-4145-96ae-0c1ea52a98de service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:04.674613 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730e79-d1f7-4145-96ae-0c1ea52a98de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:04.690217 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730e79-d1f7-4145-96ae-0c1ea52a98de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:04.692094 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730e79-d1f7-4145-96ae-0c1ea52a98de service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:04.703717 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1441/2859] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:45:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 03:45:05.467743 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476632-689d-414d-9afb-2f9b31cc981e None None] REQUEST_METHOD: `POST` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:05.468778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476632-689d-414d-9afb-2f9b31cc981e None None] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:05.469070 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476632-689d-414d-9afb-2f9b31cc981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:05.494336 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.auth.core [None req-92476632-689d-414d-9afb-2f9b31cc981e None None] MFA Rules not processed for user `e619c4b1fa304a8a99de2f60d1b6cc89`. Rule list: `[]` (Enabled: `True`). {{(pid=80355) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:45:05.495578 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-92476632-689d-414d-9afb-2f9b31cc981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:05.503442 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1419/2860] 10.222.0.41 () {58 vars in 967 bytes} [Sat May 28 03:45:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:45:05.516850 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730e79-d1f7-4145-96ae-0c1ea52a98de service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:05.528616 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730e79-d1f7-4145-96ae-0c1ea52a98de service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:05.534317 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7456e-8c41-460f-9906-131a436e52b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2f14380f60f4fa18ed6a9579cfb071b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:05.535365 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7456e-8c41-460f-9906-131a436e52b3 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:05.535856 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7456e-8c41-460f-9906-131a436e52b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:05.536321 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7456e-8c41-460f-9906-131a436e52b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:05.546639 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7456e-8c41-460f-9906-131a436e52b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:05.548592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7456e-8c41-460f-9906-131a436e52b3 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:05.559120 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1442/2861] 10.222.0.41 () {60 vars in 1384 bytes} [Sat May 28 03:45:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 03:45:06.174494 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476632-689d-414d-9afb-2f9b31cc981e None None] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:06.175988 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-92476632-689d-414d-9afb-2f9b31cc981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:06.222634 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=100 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:06.251850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476632-689d-414d-9afb-2f9b31cc981e None None] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:06.251850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:06.251850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:06.251850 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:06.252222 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] PATH_INFO: `/v3/users/c1a9600bad844f60937aa35126cd1be4` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:06.253114 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a9600bad844f60937aa35126cd1be4)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:06.255052 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:06.434044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a9600bad844f60937aa35126cd1be4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:06.452741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a9600bad844f60937aa35126cd1be4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:06.453778 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a9600bad844f60937aa35126cd1be4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:06.454510 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a9600bad844f60937aa35126cd1be4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:06.551252 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a9600bad844f60937aa35126cd1be4'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:06.564044 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1420/2862] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:45:06 2022] DELETE /identity/v3/users/c1a9600bad844f60937aa35126cd1be4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:45:06.920459 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7456e-8c41-460f-9906-131a436e52b3 service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:06.937812 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7456e-8c41-460f-9906-131a436e52b3 service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:07.002329 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:07.002496 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:07.002751 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:07.003029 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] PATH_INFO: `/v3/projects/23f8194639b14942b36011698807b031` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:07.004214 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f8194639b14942b36011698807b031)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:07.006122 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:07.035023 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:07.124160 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f8194639b14942b36011698807b031', 'request_id': 'req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.131922 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f8194639b14942b36011698807b031', 'request_id': 'req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.140971 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f8194639b14942b36011698807b031', 'request_id': 'req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.225433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] The token cache is being invalidate because project 23f8194639b14942b36011698807b031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:45:07.225722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.226630 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.228054 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1443/2863] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:45:06 2022] DELETE /identity/v3/projects/23f8194639b14942b36011698807b031 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:45:07.232118 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:07.234091 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:07.248125 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ea641-13d6-4391-90ec-25b76099df29 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:07.305806 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:07.306684 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:07.306964 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:07.307238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] PATH_INFO: `/v3/users/a61e06424d9d459286e0c33c26a534e5` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:07.308407 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a61e06424d9d459286e0c33c26a534e5)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:07.310441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:07.479163 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a61e06424d9d459286e0c33c26a534e5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.489543 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:07.492503 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a61e06424d9d459286e0c33c26a534e5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.493321 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a61e06424d9d459286e0c33c26a534e5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.494086 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a61e06424d9d459286e0c33c26a534e5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.568211 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a61e06424d9d459286e0c33c26a534e5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:07.586786 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1421/2864] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:45:07 2022] DELETE /identity/v3/users/a61e06424d9d459286e0c33c26a534e5 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:45:07.884647 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:07.886494 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:07.901719 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0bdcc-5c02-4fc3-be97-6f02d5ab7e45 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:07.972028 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:07.973403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:07.973403 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:07.973614 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] PATH_INFO: `/v3/projects/4e6febafbcf34c60bb33ade27a42b83e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:07.974707 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e6febafbcf34c60bb33ade27a42b83e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:07.977286 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:08.277969 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e6febafbcf34c60bb33ade27a42b83e', 'request_id': 'req-8236ee6c-a3c8-41fd-897e-d3ea136480de'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.284216 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e6febafbcf34c60bb33ade27a42b83e', 'request_id': 'req-8236ee6c-a3c8-41fd-897e-d3ea136480de'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.289748 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e6febafbcf34c60bb33ade27a42b83e', 'request_id': 'req-8236ee6c-a3c8-41fd-897e-d3ea136480de'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.379532 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] The token cache is being invalidate because project 4e6febafbcf34c60bb33ade27a42b83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:45:08.379819 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.380516 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8236ee6c-a3c8-41fd-897e-d3ea136480de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.382152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1444/2865] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:45:07 2022] DELETE /identity/v3/projects/4e6febafbcf34c60bb33ade27a42b83e => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:45:08.495837 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:08.497973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:08.515225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0557e-2e84-468b-82b3-e78439792106 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:08.583208 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:08.584010 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:08.584293 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:08.584571 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] PATH_INFO: `/v3/users/20b48a7b54a84c569a1ba39148dc5e70` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:08.586431 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b48a7b54a84c569a1ba39148dc5e70)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:08.588404 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:08.814789 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b48a7b54a84c569a1ba39148dc5e70'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.829451 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b48a7b54a84c569a1ba39148dc5e70'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.830627 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b48a7b54a84c569a1ba39148dc5e70'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.830813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b48a7b54a84c569a1ba39148dc5e70'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.917130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b48a7b54a84c569a1ba39148dc5e70'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:08.930484 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1422/2866] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:45:08 2022] DELETE /identity/v3/users/20b48a7b54a84c569a1ba39148dc5e70 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:45:08.935148 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50f9b-3195-43f7-82aa-f1230fca327f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:08.937736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50f9b-3195-43f7-82aa-f1230fca327f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:08.937736 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc50f9b-3195-43f7-82aa-f1230fca327f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:08.955059 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7cc50f9b-3195-43f7-82aa-f1230fca327f None None] MFA Rules not processed for user `f454c88a6bba4ab9ba311bbbd1eb0425`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:45:08.956284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7cc50f9b-3195-43f7-82aa-f1230fca327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:08.988785 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:09.014398 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1445/2867] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:45:09.024250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:09.025738 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:09.081663 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14a0d1-0050-49b6-aee9-afd3ccc684d2 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:09.085259 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2da68e-789b-4a3f-ba3d-4197b220b800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:09.085923 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2da68e-789b-4a3f-ba3d-4197b220b800 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:09.086202 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2da68e-789b-4a3f-ba3d-4197b220b800 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:09.086536 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2da68e-789b-4a3f-ba3d-4197b220b800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:09.095295 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2da68e-789b-4a3f-ba3d-4197b220b800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:09.100576 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2da68e-789b-4a3f-ba3d-4197b220b800 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:09.102045 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:09.106995 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1423/2868] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:45:09.353692 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc50f9b-3195-43f7-82aa-f1230fca327f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:09.355178 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7cc50f9b-3195-43f7-82aa-f1230fca327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:09.374504 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc50f9b-3195-43f7-82aa-f1230fca327f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:09.380446 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:09.382323 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:09.382714 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:09.383169 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] PATH_INFO: `/v3/projects/282a34892e974900926a37a76690258c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:09.384644 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=282a34892e974900926a37a76690258c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:09.387099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:09.495767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282a34892e974900926a37a76690258c', 'request_id': 'req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:09.502372 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282a34892e974900926a37a76690258c', 'request_id': 'req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:09.508556 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282a34892e974900926a37a76690258c', 'request_id': 'req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:09.592419 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] The token cache is being invalidate because project 282a34892e974900926a37a76690258c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:45:09.592659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:09.593989 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:45:09.595586 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1446/2869] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:45:09 2022] DELETE /identity/v3/projects/282a34892e974900926a37a76690258c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:45:19.587852 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:45:29.953348 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2da68e-789b-4a3f-ba3d-4197b220b800 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:45:29.962150 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2da68e-789b-4a3f-ba3d-4197b220b800 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:45:30.009175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:45:30.009813 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:45:30.010095 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:45:30.010364 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:45:30.012146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:45:30.075117 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:45:30.076557 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:45:30.086203 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1424/2870] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:45:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:45:40.081748 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=50 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:46:52.470766 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:46:52.472181 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:46:52.528595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba6123-cdf5-4fa0-8e1f-0e1290b7a296 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:46:52.532157 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd487dc-a3d2-4adc-8beb-926ac17fa5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:46:52.532787 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd487dc-a3d2-4adc-8beb-926ac17fa5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:46:52.533070 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd487dc-a3d2-4adc-8beb-926ac17fa5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:46:52.533470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd487dc-a3d2-4adc-8beb-926ac17fa5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:46:52.535256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2fd487dc-a3d2-4adc-8beb-926ac17fa5a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:46:52.592439 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd487dc-a3d2-4adc-8beb-926ac17fa5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:46:52.593915 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd487dc-a3d2-4adc-8beb-926ac17fa5a3 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:46:52.602862 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1447/2871] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:46:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:46:55.929188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:46:55.930526 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:46:55.941692 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a13b159-dbdf-4090-9a04-f6170f1f3b24 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:46:55.945269 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:46:55.945907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:46:55.946186 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:46:55.946457 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:46:55.948146 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:46:55.957984 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:46:55.959392 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:46:55.967450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1425/2872] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:46:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:47:00.897878 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227853c3-7efa-4648-8aa1-ee3631609dbd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:47:00.898424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227853c3-7efa-4648-8aa1-ee3631609dbd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:47:00.898424 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227853c3-7efa-4648-8aa1-ee3631609dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:47:00.915526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-227853c3-7efa-4648-8aa1-ee3631609dbd None None] MFA Rules not processed for user `2b515cebf18643c38f1cfc2e3fad1db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 03:47:00.916722 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-227853c3-7efa-4648-8aa1-ee3631609dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:00.923648 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1448/2873] 10.222.0.41 () {58 vars in 957 bytes} [Sat May 28 03:47:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 03:47:00.961998 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:47:00.963450 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:00.975069 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d34c3-d11d-4058-8b4d-a883e536e36e service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:47:00.978660 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7092a719870d4b5eafec8d4c51226b65', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:47:00.982175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:47:00.982175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:47:00.982175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:47:00.982175 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:01.062106 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:47:01.063493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:47:01.074590 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1426/2874] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:47:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:47:02.041330 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227853c3-7efa-4648-8aa1-ee3631609dbd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:47:02.051284 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227853c3-7efa-4648-8aa1-ee3631609dbd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:47:02.054821 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:47:02.055474 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:47:02.055757 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:47:02.056033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:47:02.057816 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:02.072521 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:47:02.073984 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:47:02.086654 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1449/2875] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:47:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:47:03.035456 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:47:03.036863 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:03.104444 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e29e1-8466-4675-a515-72d17ecd6484 service cinder] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:47:03.109462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:47:03.110130 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:47:03.110415 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:47:03.110694 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:47:03.112455 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:03.124713 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:47:03.126309 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:47:03.136068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1427/2876] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:47:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:47:09.556233 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:47:09.561361 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:09.572924 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692e718-2568-46c2-95a5-3c437d4905b8 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:47:09.586974 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:47:09.587729 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:47:09.588088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:47:09.588505 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:47:09.590438 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:09.603670 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:47:09.605624 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:47:09.617764 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1450/2877] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:47:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:47:13.131810 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=72 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:19.612962 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=63 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:19.850065 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:47:19.862441 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c2537-fc32-4b28-a573-03daa9ddce42 service placement] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:47:19.869062 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:47:19.869250 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:47:19.869448 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:47:19.869728 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:47:19.871490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:19.883867 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:47:19.885511 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:47:19.896323 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1428/2878] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:47:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:47:29.891708 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:42.378058 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:47:42.415951 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898764ca-3dc6-4384-9dfa-8641dbb0fcb8 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:47:42.419868 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:47:42.420526 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:47:42.420814 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:47:42.421121 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:47:42.422914 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:42.455433 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:47:42.456867 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:47:42.465010 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1451/2879] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:47:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:47:42.539883 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:47:42.559705 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0447481-6d2f-482e-9115-1aa0c417da9f service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:47:42.563236 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:47:42.563873 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:47:42.564159 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:47:42.564439 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:47:42.566238 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:47:42.600285 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:47:42.602281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:47:42.613166 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1429/2880] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:47:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:47:52.461280 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=27 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:47:52.608257 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=20 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:49:00.972358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:00.973808 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:01.017358 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2c951-e65d-4c60-8b49-68b6e17544d7 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:01.022549 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:01.023190 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:01.023477 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:01.023761 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:01.025575 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:01.036996 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:01.040089 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:01.049191 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1452/2881] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:49:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:49:06.562449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:06.564307 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:06.577875 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe34d1-1b8b-4f9d-bb6d-cb4f7af9cae6 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:06.581542 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd9a0e75308460080902d1da74074c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:06.582184 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:06.582466 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:06.582741 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:06.584493 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:06.668860 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:06.670349 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:06.689513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1430/2882] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:49:09.895561 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:09.906889 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf76dc-da8d-4929-b522-465bd16df363 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:09.913998 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:09.914734 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:09.915130 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:09.915484 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:09.917585 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:09.930977 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:09.932592 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:09.943007 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1453/2883] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:49:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:49:11.594831 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:11.659284 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7742d7-c286-4126-b3cf-7cb41097f8c5 service neutron] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:11.675304 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f7ebbd8eb36c4919aedd3498bd8bc483', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:11.675963 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:11.676248 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:11.676528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:11.680513 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:11.736085 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:11.738490 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:11.752696 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1431/2884] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:49:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:49:13.762882 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:13.765202 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:13.782397 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0eaed-a576-49f1-8ee7-d8633fbf64e8 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:13.787325 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:13.788227 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:13.788520 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:13.788809 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] PATH_INFO: `/v3/users/2b515cebf18643c38f1cfc2e3fad1db4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:13.790082 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b515cebf18643c38f1cfc2e3fad1db4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:13.792220 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:14.127033 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b515cebf18643c38f1cfc2e3fad1db4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.133925 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b515cebf18643c38f1cfc2e3fad1db4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.146744 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b515cebf18643c38f1cfc2e3fad1db4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.147469 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b515cebf18643c38f1cfc2e3fad1db4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.226595 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b515cebf18643c38f1cfc2e3fad1db4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.233093 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1454/2885] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:49:13 2022] DELETE /identity/v3/users/2b515cebf18643c38f1cfc2e3fad1db4 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:49:14.589927 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:14.607876 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a46ee0-9fc3-4b59-ac38-af4577fb0e15 service nova] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:14.671529 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:14.672410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:14.672770 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:14.673008 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] PATH_INFO: `/v3/projects/c0fe831f6e954e05b7d448f1d4f43754` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:14.674244 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0fe831f6e954e05b7d448f1d4f43754)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:14.676283 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:14.770347 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:49:14.787350 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0fe831f6e954e05b7d448f1d4f43754', 'request_id': 'req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.796077 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0fe831f6e954e05b7d448f1d4f43754', 'request_id': 'req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.802056 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0fe831f6e954e05b7d448f1d4f43754', 'request_id': 'req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.874409 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] The token cache is being invalidate because project c0fe831f6e954e05b7d448f1d4f43754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:49:14.874649 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.875528 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:14.877071 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1432/2886] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:49:14 2022] DELETE /identity/v3/projects/c0fe831f6e954e05b7d448f1d4f43754 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:49:14.881631 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:14.885258 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:14.898046 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e879-59ba-4dc8-a754-77384395f1d9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:14.957308 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:14.959815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:14.959815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:14.959815 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] PATH_INFO: `/v3/users/485a901fccf7445bbc0f27eaeec33ad0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:14.960208 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] RBAC: Authorizing `identity:delete_user(user_id=485a901fccf7445bbc0f27eaeec33ad0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:14.961845 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:15.077043 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:49:15.199921 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '485a901fccf7445bbc0f27eaeec33ad0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.207354 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '485a901fccf7445bbc0f27eaeec33ad0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.218827 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '485a901fccf7445bbc0f27eaeec33ad0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.221315 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '485a901fccf7445bbc0f27eaeec33ad0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.343071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '485a901fccf7445bbc0f27eaeec33ad0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.350075 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1455/2887] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:49:14 2022] DELETE /identity/v3/users/485a901fccf7445bbc0f27eaeec33ad0 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:49:15.578967 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:15.580440 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:15.593973 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fdb9c-e7d4-4f88-82cc-2514dc845df5 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:15.658083 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:15.659122 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:15.659410 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:15.659690 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] PATH_INFO: `/v3/projects/dbf723c339db4b60abd1318b96b6b1fa` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:15.661074 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf723c339db4b60abd1318b96b6b1fa)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:15.663209 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:15.800829 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf723c339db4b60abd1318b96b6b1fa', 'request_id': 'req-44a37297-3077-4bf9-b531-65e8051195c3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.806977 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf723c339db4b60abd1318b96b6b1fa', 'request_id': 'req-44a37297-3077-4bf9-b531-65e8051195c3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.812920 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf723c339db4b60abd1318b96b6b1fa', 'request_id': 'req-44a37297-3077-4bf9-b531-65e8051195c3'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.906911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] The token cache is being invalidate because project dbf723c339db4b60abd1318b96b6b1fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80355) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:49:15.906911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.906911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:15.906911 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1433/2888] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:49:15 2022] DELETE /identity/v3/projects/dbf723c339db4b60abd1318b96b6b1fa => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 03:49:20.703142 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:20.703860 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:20.775870 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eec74-d7cb-480f-b851-9d452fdab35a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:20.840807 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:20.842021 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:20.842305 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:20.842625 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:20.844475 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:20.855880 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:20.858088 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:20.871908 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1456/2889] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:49:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:49:24.166954 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:24.169225 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:24.219910 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:49:24.235111 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a37297-3077-4bf9-b531-65e8051195c3 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:24.239772 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:24.240647 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:24.240934 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:24.241255 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] PATH_INFO: `/v3/users/303285e036734e5cbd491920ae7473d9` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:24.242449 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303285e036734e5cbd491920ae7473d9)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:24.244382 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:24.430843 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303285e036734e5cbd491920ae7473d9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:24.443357 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303285e036734e5cbd491920ae7473d9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:24.444030 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303285e036734e5cbd491920ae7473d9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:24.444907 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303285e036734e5cbd491920ae7473d9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:24.508538 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303285e036734e5cbd491920ae7473d9'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:24.522224 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1434/2890] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:49:24 2022] DELETE /identity/v3/users/303285e036734e5cbd491920ae7473d9 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:49:24.804775 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:24.816402 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05b6d9-386e-4eac-86b8-a7a016b4fba9 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:24.852289 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:49:24.875634 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:24.876481 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:24.876767 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:24.877042 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] PATH_INFO: `/v3/projects/d1d9f51a3cc44062af827a498842ee80` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:24.878253 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d9f51a3cc44062af827a498842ee80)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:24.880152 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:24.969811 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d9f51a3cc44062af827a498842ee80', 'request_id': 'req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:24.975935 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d9f51a3cc44062af827a498842ee80', 'request_id': 'req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:24.981248 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d9f51a3cc44062af827a498842ee80', 'request_id': 'req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.036499 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] The token cache is being invalidate because project d1d9f51a3cc44062af827a498842ee80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:49:25.036817 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.037452 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.039071 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1457/2891] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:49:24 2022] DELETE /identity/v3/projects/d1d9f51a3cc44062af827a498842ee80 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:49:25.043068 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:25.044474 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:25.057462 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d8a13-3a64-4918-9f92-78e24f2c2fe9 admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:25.113898 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:25.114768 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:25.115088 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:25.115362 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] PATH_INFO: `/v3/users/286dcf1a1a1c4a2ea8bc8a8d79c4d914` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:25.116569 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=286dcf1a1a1c4a2ea8bc8a8d79c4d914)` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:25.118522 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:25.313993 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286dcf1a1a1c4a2ea8bc8a8d79c4d914'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.325352 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286dcf1a1a1c4a2ea8bc8a8d79c4d914'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.326580 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286dcf1a1a1c4a2ea8bc8a8d79c4d914'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.327188 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286dcf1a1a1c4a2ea8bc8a8d79c4d914'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.329672 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:49:25.405444 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.notifications [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286dcf1a1a1c4a2ea8bc8a8d79c4d914'} {{(pid=80355) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.417895 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1435/2892] 10.222.0.41 () {58 vars in 1223 bytes} [Sat May 28 03:49:25 2022] DELETE /identity/v3/users/286dcf1a1a1c4a2ea8bc8a8d79c4d914 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:49:25.688126 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:49:25.689753 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:49:25.704411 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf2ae2-6f7a-4e1a-befe-67a8de8a1c67 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:49:25.761612 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f454c88a6bba4ab9ba311bbbd1eb0425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc9d398b29a149a4ab99273f43b67fb2', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:49:25.762470 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:49:25.762749 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:49:25.763024 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] PATH_INFO: `/v3/projects/fad1092444f04842bf6eee17a36bf043` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:49:25.764176 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fad1092444f04842bf6eee17a36bf043)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:49:25.766099 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:49:25.862256 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fad1092444f04842bf6eee17a36bf043', 'request_id': 'req-1420688b-3eef-40eb-918a-264788c97abb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.868378 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fad1092444f04842bf6eee17a36bf043', 'request_id': 'req-1420688b-3eef-40eb-918a-264788c97abb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.874511 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fad1092444f04842bf6eee17a36bf043', 'request_id': 'req-1420688b-3eef-40eb-918a-264788c97abb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.966659 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:49:25.971392 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] The token cache is being invalidate because project fad1092444f04842bf6eee17a36bf043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 03:49:25.971683 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.975794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1420688b-3eef-40eb-918a-264788c97abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 03:49:25.975794 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1458/2893] 10.222.0.41 () {58 vars in 1232 bytes} [Sat May 28 03:49:25 2022] DELETE /identity/v3/projects/fad1092444f04842bf6eee17a36bf043 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 03:49:35.468419 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:DELETE=1,keystone:SELECT=6 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:49:36.079675 dv-cinder-843361-3-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 03:54:03.639109 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] Authenticating user token {{(pid=80355) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 03:54:03.640531 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:54:03.708541 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4fe60-dbb3-4d2e-bd34-6658820b5a7c admin admin] Validating token access rules against request {{(pid=80355) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 03:54:03.765838 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0578acf-130a-4f80-bd06-479696bc793a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1e771dc83fa34d79947647e56ca338a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb46e722ef6478eb40cec80b8278e24', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80355) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 03:54:03.766476 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578acf-130a-4f80-bd06-479696bc793a service placement] REQUEST_METHOD: `GET` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 03:54:03.766759 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578acf-130a-4f80-bd06-479696bc793a service placement] SCRIPT_NAME: `/identity` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 03:54:03.767036 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578acf-130a-4f80-bd06-479696bc793a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80355) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 03:54:03.768809 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.fernet_utils [None req-c0578acf-130a-4f80-bd06-479696bc793a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80355) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 03:54:03.780281 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0578acf-130a-4f80-bd06-479696bc793a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 03:54:03.781896 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0578acf-130a-4f80-bd06-479696bc793a service placement] RBAC: Authorization granted {{(pid=80355) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 03:54:03.791524 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: [pid: 80355|app: 0|req: 1436/2894] 10.222.0.41 () {60 vars in 1365 bytes} [Sat May 28 03:54:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 03:54:13.787340 dv-cinder-843361-3-iscsi devstack@keystone.service[80355]: DEBUG dbcounter [-] [80355] Writing DB stats keystone:SELECT=58 {{(pid=80355) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}